[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:11 dist-test-slave-69rg krb5kdc[30284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:11 dist-test-slave-69rg krb5kdc[30284](info): set up 2 sockets
Aug 11 11:10:11 dist-test-slave-69rg krb5kdc[30284](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 11:10:12.396973 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:12 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:12.546347 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42521
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:12.638549 30297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:12.638761 30297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:12.638785 30297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:12.639868 30297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:12.639909 30297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:12.639925 30297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:12.639940 30297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:12.639952 30297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:12.641439 30297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:12.641646 30297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:12.641860 30297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:12.648692 30303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:12.649174 30305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:12.649199 30297 server_base.cc:1034] running on GCE node
W20260811 11:10:12.649183 30302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:12.649571 30297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:12.649957 30297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:12.651165 30297 hybrid_clock.cc:648] HybridClock initialized: now 1786446612651173 us; error 54 us; skew 500 ppm
Aug 11 11:10:12 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:12.653754 30297 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:12.654220 30297 webserver.cc:460] Webserver started at http://127.29.145.62:35777/ using document root <none> and password file <none>
I20260811 11:10:12.654474 30297 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:12.654542 30297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:12.654690 30297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:12.656786 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "bb5b976252b74f2f94185b1db5b3d063"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "aa2f6c89a9fce0743a25482c22999f2f"
server_key_iv: "e6d531688fb93a58eb90733dc5c6d8ae"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.657171 30297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "bb5b976252b74f2f94185b1db5b3d063"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "aa2f6c89a9fce0743a25482c22999f2f"
server_key_iv: "e6d531688fb93a58eb90733dc5c6d8ae"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.658785 30297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:12.659789 30312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:12.660053 30297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:12.660189 30297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "bb5b976252b74f2f94185b1db5b3d063"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "aa2f6c89a9fce0743a25482c22999f2f"
server_key_iv: "e6d531688fb93a58eb90733dc5c6d8ae"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.660305 30297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:12.674414 30297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:12.676837 30297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:12.677016 30297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:12.681722 30297 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42521
I20260811 11:10:12.681766 30366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42521 every 8 connection(s)
I20260811 11:10:12.682192 30297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:12.683107 30367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:12.684376 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30297
I20260811 11:10:12.684502 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:12.685320 30276 external_mini_cluster.cc:1428] Setting key 800546a383d6ca5e100f620608b3b505
I20260811 11:10:12.687207 30367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063: Bootstrap starting.
I20260811 11:10:12.688021 30367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:12.688397 30367 log.cc:824] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:12.689411 30367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063: No bootstrap required, opened a new log
I20260811 11:10:12.691897 30367 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } }
I20260811 11:10:12.692060 30367 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:12.692112 30367 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb5b976252b74f2f94185b1db5b3d063, State: Initialized, Role: FOLLOWER
I20260811 11:10:12.692251 30367 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } }
I20260811 11:10:12.692337 30367 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:12.692377 30367 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:12.692428 30367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:12.693671 30367 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } }
I20260811 11:10:12.693786 30367 leader_election.cc:302] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb5b976252b74f2f94185b1db5b3d063; no voters:
I20260811 11:10:12.693980 30367 leader_election.cc:288] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:12.694044 30372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:12.694267 30372 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [term 1 LEADER]: Becoming Leader. State: Replica: bb5b976252b74f2f94185b1db5b3d063, State: Running, Role: LEADER
I20260811 11:10:12.694270 30367 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:12.694382 30372 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } }
I20260811 11:10:12.694749 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb5b976252b74f2f94185b1db5b3d063" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } } }
I20260811 11:10:12.694762 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb5b976252b74f2f94185b1db5b3d063. Latest consensus state: current_term: 1 leader_uuid: "bb5b976252b74f2f94185b1db5b3d063" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb5b976252b74f2f94185b1db5b3d063" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42521 } } }
I20260811 11:10:12.694857 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:12.695070 30379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:12.694861 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:12.695631 30379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:12.697510 30379 catalog_manager.cc:1347] Generated new cluster ID: be459bd344e348139cfc25b1a9d2ae23
I20260811 11:10:12.697567 30379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:12 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:12.704380 30379 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:12.705017 30379 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:12.707047 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:12.687299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60206 (local address 127.29.145.62:42521)
0811 11:10:12.687597 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:12.687610 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:12.688025 (+ 415us) server_negotiation.cc:407] Connection header received
0811 11:10:12.691062 (+ 3037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:12.691071 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:12.691140 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:12.691298 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:12.693123 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.694581 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.696266 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.696761 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.696891 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:12.701561 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:12.701566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:12.701566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:12.701578 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:12.704742 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:12.705342 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:12.705346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:12.705346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:12.705445 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:12.705762 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:12.705763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:12.705763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:12.706046 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:12.706353 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:12.706590 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 11:10:12.706732 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":141,"threads_started":1}
I20260811 11:10:12.712742 30379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb5b976252b74f2f94185b1db5b3d063: Generated new TSK 0
I20260811 11:10:12.712916 30379 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:12.746045 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:12.848320 30394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:12.848618 30394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:12.848675 30394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:12.850093 30394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:12.850171 30394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:12.850220 30394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:12.851985 30394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:12.852329 30394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:12.852591 30394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:12.858881 30401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:12.859051 30394 server_base.cc:1034] running on GCE node
W20260811 11:10:12.858958 30404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:12.858927 30400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:12.859447 30394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:12.859750 30394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:12.860941 30394 hybrid_clock.cc:648] HybridClock initialized: now 1786446612860928 us; error 30 us; skew 500 ppm
Aug 11 11:10:12 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:12.863972 30394 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:12.864511 30394 webserver.cc:460] Webserver started at http://127.29.145.1:34763/ using document root <none> and password file <none>
I20260811 11:10:12.864773 30394 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:12.864843 30394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:12.864993 30394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:12.866206 30394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "882d00e38116495db645bbeac5fb2957"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "729cdfc736ec88620e10cf600c581b39"
server_key_iv: "6d4ea11f07b25d4d8ddb6b5d10d5b05d"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.866601 30394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "882d00e38116495db645bbeac5fb2957"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "729cdfc736ec88620e10cf600c581b39"
server_key_iv: "6d4ea11f07b25d4d8ddb6b5d10d5b05d"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.868016 30394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:12.868891 30413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:12.869117 30394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:12.869230 30394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "882d00e38116495db645bbeac5fb2957"
format_stamp: "Formatted at 2026-08-11 11:10:12 on dist-test-slave-69rg"
server_key: "729cdfc736ec88620e10cf600c581b39"
server_key_iv: "6d4ea11f07b25d4d8ddb6b5d10d5b05d"
server_key_version: "encryptionkey@0"
I20260811 11:10:12.869336 30394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:12.882625 30394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:12.885306 30394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:12.885502 30394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:12.885952 30394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:12.886350 30394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:12.886415 30394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:12.886451 30394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:12.886482 30394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:12.892970 30394 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:45593
I20260811 11:10:12.893064 30538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:45593 every 8 connection(s)
I20260811 11:10:12.893399 30394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:12 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446612, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:12.902940 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30394
I20260811 11:10:12.903049 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:12.903223 30276 external_mini_cluster.cc:1428] Setting key 58b6f5ed1cc6a248243ae54a26723113
I20260811 11:10:12.903290 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:12.894353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:49555 (local address 127.29.145.62:42521)
0811 11:10:12.894419 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:12.894422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:12.894714 (+ 292us) server_negotiation.cc:407] Connection header received
0811 11:10:12.895267 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:12.895269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:12.895281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:12.895357 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:12.896428 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.896947 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.897950 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.898167 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.898226 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:12.900272 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:12.900273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:12.900274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:12.900280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:12.901673 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:12.902217 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:12.902218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:12.902218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:12.902287 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:12.902531 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:12.902532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:12.902532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:12.902639 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:12.902739 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:12.903057 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 11:10:12.903154 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:12.903416 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:12.894389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42521 (local address 127.29.145.1:49555)
0811 11:10:12.894533 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:12.894571 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:12.895131 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:12.895469 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:12.895473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:12.896121 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:12.896323 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:12.896328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.897028 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.897029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:12.897860 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:12.897863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:12.898246 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:12.898247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:12.898351 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:12.898820 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:12.898829 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:12.900178 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:12.901750 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:12.901757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:12.901758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:12.902090 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:12.902359 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:12.902359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:12.902360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:12.902471 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:12.902777 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:12.902785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:12.902906 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 11:10:12.903035 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 11:10:12.903209 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260811 11:10:12.904013 30539 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42521
I20260811 11:10:12.904129 30539 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:12.904315 30539 heartbeater.cc:511] Master 127.29.145.62:42521 requested a full tablet report, sending...
I20260811 11:10:12.904853 30328 ts_manager.cc:194] Registered new tserver with Master: 882d00e38116495db645bbeac5fb2957 (127.29.145.1:45593)
I20260811 11:10:12.905472 30328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:49555
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:12.932696 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:13.018935 30544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:13.019158 30544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:13.019186 30544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:13.020413 30544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:13.020469 30544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:13.020499 30544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:13.022184 30544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:13.022465 30544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:13.022725 30544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:13.028157 30550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:13.028498 30549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:13.028524 30552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:13.028673 30544 server_base.cc:1034] running on GCE node
I20260811 11:10:13.028910 30544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:13.029130 30544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:13.030301 30544 hybrid_clock.cc:648] HybridClock initialized: now 1786446613030310 us; error 61 us; skew 500 ppm
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:13.033242 30544 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:13.033876 30544 webserver.cc:460] Webserver started at http://127.29.145.2:45593/ using document root <none> and password file <none>
I20260811 11:10:13.034166 30544 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:13.034216 30544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:13.034360 30544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:13.035493 30544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "90b69f0ad1124b6b823a8290e689841d"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "c60ed475b6ae5ab34139ccdc3e7d8a19"
server_key_iv: "e9b532f8ec09536b48add680542360bb"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.035861 30544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "90b69f0ad1124b6b823a8290e689841d"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "c60ed475b6ae5ab34139ccdc3e7d8a19"
server_key_iv: "e9b532f8ec09536b48add680542360bb"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.037271 30544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:13.038187 30560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.038372 30544 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:13.038467 30544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "90b69f0ad1124b6b823a8290e689841d"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "c60ed475b6ae5ab34139ccdc3e7d8a19"
server_key_iv: "e9b532f8ec09536b48add680542360bb"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.038527 30544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:13.060211 30544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:13.062660 30544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:13.062781 30544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:13.062981 30544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:13.063325 30544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:13.063360 30544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.063416 30544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:13.063447 30544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.070405 30544 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:44039
I20260811 11:10:13.070782 30685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:44039 every 8 connection(s)
I20260811 11:10:13.070842 30544 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:13.078958 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30544
I20260811 11:10:13.079042 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:13.079217 30276 external_mini_cluster.cc:1428] Setting key ec24fe5f9c8470996b13e6f61457a033
I20260811 11:10:13.081049 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.071566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42521 (local address 127.29.145.2:41905)
0811 11:10:13.071755 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:13.071790 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:13.072431 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:13.072694 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:13.072698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:13.073288 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:13.073556 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:13.073561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.074324 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.074325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:13.075142 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:13.075144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.075555 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.075557 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:13.075685 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.076229 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:13.076239 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:13.077764 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:13.079526 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:13.079532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:13.079534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:13.079853 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:13.080082 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:13.080083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:13.080083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:13.080246 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:13.080545 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:13.080550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:13.080671 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 11:10:13.080758 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 11:10:13.080883 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260811 11:10:13.081048 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.071585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41905 (local address 127.29.145.62:42521)
0811 11:10:13.071646 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:13.071649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:13.071870 (+ 221us) server_negotiation.cc:407] Connection header received
0811 11:10:13.072534 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:13.072536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:13.072548 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:13.072621 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:13.073653 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.074226 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.075239 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.075457 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.075543 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.077911 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:13.077913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:13.077913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:13.077921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:13.079435 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.079936 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.079937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.079938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.079990 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.080319 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.080320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.080320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.080475 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:13.080550 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:13.080797 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 11:10:13.080906 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:10:13.081529 30686 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42521
I20260811 11:10:13.081615 30686 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:13.081758 30686 heartbeater.cc:511] Master 127.29.145.62:42521 requested a full tablet report, sending...
I20260811 11:10:13.082154 30328 ts_manager.cc:194] Registered new tserver with Master: 90b69f0ad1124b6b823a8290e689841d (127.29.145.2:44039)
I20260811 11:10:13.082679 30328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:41905
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:13.108909 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:13.200879 30691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:13.201161 30691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:13.201216 30691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:13.202600 30691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:13.202670 30691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:13.202714 30691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:13.204334 30691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42521
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:13.204638 30691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:13.204900 30691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:13.210768 30696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:13.210788 30697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:13.210772 30699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:13.211117 30691 server_base.cc:1034] running on GCE node
I20260811 11:10:13.211360 30691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:13.211635 30691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:13.212836 30691 hybrid_clock.cc:648] HybridClock initialized: now 1786446613212820 us; error 31 us; skew 500 ppm
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:13.215677 30691 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:13.216144 30691 webserver.cc:460] Webserver started at http://127.29.145.3:46845/ using document root <none> and password file <none>
I20260811 11:10:13.216406 30691 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:13.216578 30691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:13.216755 30691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:13.217998 30691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "95c0336ba0e041b3b539e22bc8643cdd"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "2e5a3e343372101ebbb9c858a8a5bbde"
server_key_iv: "fddd42366f4d2f4163102f97ff8acbfd"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.218400 30691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "95c0336ba0e041b3b539e22bc8643cdd"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "2e5a3e343372101ebbb9c858a8a5bbde"
server_key_iv: "fddd42366f4d2f4163102f97ff8acbfd"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.219859 30691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:13.220764 30706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.221000 30691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:13.221112 30691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "95c0336ba0e041b3b539e22bc8643cdd"
format_stamp: "Formatted at 2026-08-11 11:10:13 on dist-test-slave-69rg"
server_key: "2e5a3e343372101ebbb9c858a8a5bbde"
server_key_iv: "fddd42366f4d2f4163102f97ff8acbfd"
server_key_version: "encryptionkey@0"
I20260811 11:10:13.221190 30691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:13.238210 30691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:13.240682 30691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:13.240888 30691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:13.241214 30691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:13.241567 30691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:13.241623 30691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.241675 30691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:13.241707 30691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:13.248875 30691 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:44275
I20260811 11:10:13.248939 30833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:44275 every 8 connection(s)
I20260811 11:10:13.249325 30691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:13.255540 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30691
I20260811 11:10:13.255645 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:13.255820 30276 external_mini_cluster.cc:1428] Setting key 0470141e19583a349193e272828f91f4
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:13.260785 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.250429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:40815 (local address 127.29.145.62:42521)
0811 11:10:13.250576 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:13.250579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:13.250597 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:10:13.251107 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:13.251109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:13.251122 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:13.251242 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:13.252335 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.252901 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.254102 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.254354 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.254425 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.257551 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:13.257553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:13.257553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:13.257563 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:13.259172 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.259655 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.259656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.259656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.259713 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.259995 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.259996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.259996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.260149 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:13.260204 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:13.260616 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 11:10:13.260697 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 11:10:13.260958 30836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.250245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42521 (local address 127.29.145.3:40815)
0811 11:10:13.250429 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:13.250476 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:13.250984 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:13.251321 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:13.251325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:13.251963 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:13.252172 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:13.252176 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.253014 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.253016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:13.254009 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:13.254012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.254470 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.254472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:13.254612 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.255249 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:13.255259 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:13.257428 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:13.259273 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:13.259282 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:13.259284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:13.259537 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:13.259808 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:13.259809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:13.259809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:13.259930 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:13.260235 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:13.260242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:13.260419 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 11:10:13.260535 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:10:13.260688 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260811 11:10:13.261528 30834 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42521
I20260811 11:10:13.261641 30834 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:13.261868 30834 heartbeater.cc:511] Master 127.29.145.62:42521 requested a full tablet report, sending...
I20260811 11:10:13.262274 30328 ts_manager.cc:194] Registered new tserver with Master: 95c0336ba0e041b3b539e22bc8643cdd (127.29.145.3:44275)
I20260811 11:10:13.262676 30328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:40815
I20260811 11:10:13.268668 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:13.297645 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.288370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.29.145.1:45593)
0811 11:10:13.288560 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:13.288565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:13.288575 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:13.288604 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:13.288606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:13.288621 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:13.288702 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:13.289304 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.290153 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.290957 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.291338 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.291407 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.293780 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:13.293798 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:13.293799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:13.293806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:13.296075 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.296544 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.296548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.296548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.296645 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.296890 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.296891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.296892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.297048 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:13.297173 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:13.297332 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 11:10:13.297461 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
W20260811 11:10:13.298127 30442 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54120
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:13.324457 30841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:13.316360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.29.145.1:45593)
0811 11:10:13.316465 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:13.316466 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:13.316540 (+ 74us) server_negotiation.cc:407] Connection header received
0811 11:10:13.316601 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:13.316602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:13.316613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:13.316668 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:13.317331 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.317910 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.318728 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:13.318949 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:13.319009 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:13.321529 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:13.321531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:13.321531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:13.321539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:13.322905 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.323287 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.323288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.323289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.323355 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:13.323612 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:13.323613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:13.323614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:13.323730 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:13.324011 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:13.324269 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 11:10:13.324351 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 11:10:13.325317 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30394
I20260811 11:10:13.334925 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30544
I20260811 11:10:13.342450 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30691
I20260811 11:10:13.351926 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30297
2026-08-11T11:10:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2125 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30848](info): set up 2 sockets
Aug 11 11:10:13 dist-test-slave-69rg krb5kdc[30848](info): commencing operation
krb5kdc: starting...
W20260811 11:10:14.476750 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:14 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:14.602849 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44699
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:44699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:14.687141 30862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:14.687379 30862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:14.687417 30862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:14.688544 30862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:14.688606 30862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:14.688632 30862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:14.688663 30862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:14.688697 30862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:14.690255 30862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:44699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:14.690528 30862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:14.690764 30862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:14.696020 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:14.696089 30870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:14.696035 30869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:14.696077 30862 server_base.cc:1034] running on GCE node
I20260811 11:10:14.696496 30862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:14.696800 30862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:14.697938 30862 hybrid_clock.cc:648] HybridClock initialized: now 1786446614697915 us; error 34 us; skew 500 ppm
Aug 11 11:10:14 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:14.700559 30862 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:14.701066 30862 webserver.cc:460] Webserver started at http://127.29.145.62:33215/ using document root <none> and password file <none>
I20260811 11:10:14.701294 30862 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:14.701337 30862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:14.701431 30862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:14.702517 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "ae7b834bee2e4c57945da2b0a1d8fb63"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "92a95551d4584bf7d642847167107351"
server_key_iv: "b203442b892942ca91ba5e7bea4cc299"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.702857 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "ae7b834bee2e4c57945da2b0a1d8fb63"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "92a95551d4584bf7d642847167107351"
server_key_iv: "b203442b892942ca91ba5e7bea4cc299"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.704173 30862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:14.704944 30879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:14.705132 30862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:14.705224 30862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "ae7b834bee2e4c57945da2b0a1d8fb63"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "92a95551d4584bf7d642847167107351"
server_key_iv: "b203442b892942ca91ba5e7bea4cc299"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.705305 30862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:14.721801 30862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:14.724254 30862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:14.724435 30862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:14.728874 30862 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:44699
I20260811 11:10:14.728936 30939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:44699 every 8 connection(s)
I20260811 11:10:14.729310 30862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:14.730223 30940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:14.732587 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63: Bootstrap starting.
I20260811 11:10:14.733199 30940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:14.733507 30940 log.cc:824] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:14.734380 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63: No bootstrap required, opened a new log
I20260811 11:10:14.736548 30940 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } }
I20260811 11:10:14.736687 30940 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:14.736711 30940 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7b834bee2e4c57945da2b0a1d8fb63, State: Initialized, Role: FOLLOWER
I20260811 11:10:14.736873 30940 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } }
I20260811 11:10:14.736948 30940 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:14.736999 30940 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:14.737048 30940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:14.737586 30940 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } }
I20260811 11:10:14.737694 30940 leader_election.cc:302] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae7b834bee2e4c57945da2b0a1d8fb63; no voters:
I20260811 11:10:14.737877 30940 leader_election.cc:288] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:14.737946 30943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:14.738097 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30862
I20260811 11:10:14.738128 30943 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [term 1 LEADER]: Becoming Leader. State: Replica: ae7b834bee2e4c57945da2b0a1d8fb63, State: Running, Role: LEADER
I20260811 11:10:14.738184 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:14.738193 30940 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:14.738216 30943 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } }
I20260811 11:10:14.738363 30276 external_mini_cluster.cc:1428] Setting key b8837f7bfe7261ddfc68ae5b4d3a597b
I20260811 11:10:14.738494 30943 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae7b834bee2e4c57945da2b0a1d8fb63. Latest consensus state: current_term: 1 leader_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } } }
I20260811 11:10:14.738569 30943 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:14.738654 30945 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7b834bee2e4c57945da2b0a1d8fb63" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44699 } } }
I20260811 11:10:14.738746 30945 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:14.738891 30949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:14.739468 30949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:14.742915 30949 catalog_manager.cc:1347] Generated new cluster ID: e9cb2b83145f46f4b80da1caac94047b
I20260811 11:10:14.742973 30949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:14 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:14.753455 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:14.739191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.29.145.62:44699)
0811 11:10:14.740133 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:14.740140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:14.740165 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:10:14.740251 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:14.740256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:14.740296 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:14.740437 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:14.741186 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.742166 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.743100 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.743513 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.743631 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:14.750207 (+ 6576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:14.750211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:14.750212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:14.750253 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:14.751898 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:14.752271 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:14.752274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:14.752275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:14.752365 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:14.752568 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:14.752569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:14.752570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:14.752818 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:14.753003 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:14.753075 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:10:14.753237 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"thread_start_us":84,"threads_started":1}
I20260811 11:10:14.758781 30949 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:14.759347 30949 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:14.761936 30949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae7b834bee2e4c57945da2b0a1d8fb63: Generated new TSK 0
I20260811 11:10:14.762126 30949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:14.797161 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:14.902840 30966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:14.903107 30966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:14.903133 30966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:14.904301 30966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:14.904346 30966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:14.904392 30966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:14.906001 30966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:14.906260 30966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:14.906498 30966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:14.912020 30975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:14.912051 30972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:14.912020 30973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:14.912357 30966 server_base.cc:1034] running on GCE node
I20260811 11:10:14.912536 30966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:14.912823 30966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:14.913987 30966 hybrid_clock.cc:648] HybridClock initialized: now 1786446614913968 us; error 58 us; skew 500 ppm
Aug 11 11:10:14 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:14.916648 30966 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:14.917142 30966 webserver.cc:460] Webserver started at http://127.29.145.1:43691/ using document root <none> and password file <none>
I20260811 11:10:14.917397 30966 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:14.917464 30966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:14.917610 30966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:14.918751 30966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "793e9d1055f04fdebb3f328725464861"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "23417e9091a19d8235f40b046c65f6e4"
server_key_iv: "32d074750cba07c76d93405b45ad3a97"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.919112 30966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "793e9d1055f04fdebb3f328725464861"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "23417e9091a19d8235f40b046c65f6e4"
server_key_iv: "32d074750cba07c76d93405b45ad3a97"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.920518 30966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:14.921337 30985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:14.921581 30966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:14.921687 30966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "793e9d1055f04fdebb3f328725464861"
format_stamp: "Formatted at 2026-08-11 11:10:14 on dist-test-slave-69rg"
server_key: "23417e9091a19d8235f40b046c65f6e4"
server_key_iv: "32d074750cba07c76d93405b45ad3a97"
server_key_version: "encryptionkey@0"
I20260811 11:10:14.921785 30966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:14.947254 30966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:14.949431 30966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:14.949602 30966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:14.949894 30966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:14.950240 30966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:14.950295 30966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:14.950333 30966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:14.950361 30966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:14.956574 30966 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:45673
I20260811 11:10:14.956619 31107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:45673 every 8 connection(s)
I20260811 11:10:14.956951 30966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:14 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:14.963927 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 30966
I20260811 11:10:14.964028 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:14.964212 30276 external_mini_cluster.cc:1428] Setting key 096b54babb8bb7a81fde212e464fdcce
I20260811 11:10:14.967132 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:14.957702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:51487 (local address 127.29.145.62:44699)
0811 11:10:14.957788 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:14.957791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:14.958006 (+ 215us) server_negotiation.cc:407] Connection header received
0811 11:10:14.959154 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:14.959155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:14.959169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:14.959218 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:14.959881 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.960411 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.961443 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.961666 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.961743 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:14.964109 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:14.964111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:14.964111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:14.964119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:14.965684 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:14.966132 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:14.966134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:14.966134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:14.966194 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:14.966466 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:14.966467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:14.966467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:14.966581 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:14.966649 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:14.966943 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 11:10:14.967040 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:10:14.967303 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:14.957652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44699 (local address 127.29.145.1:51487)
0811 11:10:14.957863 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:14.957917 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:14.959027 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:14.959316 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:14.959325 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:14.959585 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:14.959782 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:14.959787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.960512 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.960513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:14.961348 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:14.961350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:14.961752 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:14.961753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:14.961957 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:14.962513 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:14.962522 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:14.963999 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:14.965786 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:14.965795 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:14.965797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:14.966053 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:14.966272 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:14.966273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:14.966274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:14.966393 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:14.966654 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:14.966662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:14.966805 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 11:10:14.966905 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 11:10:14.967062 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260811 11:10:14.967818 31108 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44699
I20260811 11:10:14.967932 31108 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:14.968132 31108 heartbeater.cc:511] Master 127.29.145.62:44699 requested a full tablet report, sending...
I20260811 11:10:14.968717 30900 ts_manager.cc:194] Registered new tserver with Master: 793e9d1055f04fdebb3f328725464861 (127.29.145.1:45673)
I20260811 11:10:14.969466 30900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:51487
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:14.993019 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:15.085636 31113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:15.085928 31113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:15.085974 31113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:15.087255 31113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:15.087327 31113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:15.087375 31113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:15.088943 31113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:15.089239 31113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:15.089483 31113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:15.095476 31120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:15.095527 31119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:15.095510 31124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:15.096201 31113 server_base.cc:1034] running on GCE node
I20260811 11:10:15.096498 31113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:15.096772 31113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:15.097892 31113 hybrid_clock.cc:648] HybridClock initialized: now 1786446615097898 us; error 53 us; skew 500 ppm
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:15.100639 31113 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:15.101151 31113 webserver.cc:460] Webserver started at http://127.29.145.2:33817/ using document root <none> and password file <none>
I20260811 11:10:15.101426 31113 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:15.101495 31113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:15.101647 31113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:15.102814 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "30edf72f60604dc4965e0e4010726e4f"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "d1cd406ecd4fc2c5f5ea392a28b31a7b"
server_key_iv: "87a88a0e709403abd36cd56965ae9d56"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.103202 31113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "30edf72f60604dc4965e0e4010726e4f"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "d1cd406ecd4fc2c5f5ea392a28b31a7b"
server_key_iv: "87a88a0e709403abd36cd56965ae9d56"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.104586 31113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:15.105474 31133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.105759 31113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:15.105917 31113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "30edf72f60604dc4965e0e4010726e4f"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "d1cd406ecd4fc2c5f5ea392a28b31a7b"
server_key_iv: "87a88a0e709403abd36cd56965ae9d56"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.106000 31113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:15.118839 31113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:15.121035 31113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:15.121238 31113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:15.121510 31113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:15.121899 31113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:15.121956 31113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.122000 31113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:15.122044 31113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.128607 31113 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:34309
I20260811 11:10:15.128650 31261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:34309 every 8 connection(s)
I20260811 11:10:15.128989 31113 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:15.139138 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31113
I20260811 11:10:15.139231 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.129952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41107 (local address 127.29.145.62:44699)
0811 11:10:15.130008 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.130010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.130209 (+ 199us) server_negotiation.cc:407] Connection header received
0811 11:10:15.131246 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.131247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.131262 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.131345 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.132005 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.132522 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.133564 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.133772 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.133925 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.136308 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.136310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.136310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.136318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.137743 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.138191 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.138192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.138193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.138257 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.138521 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.138522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.138523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.138652 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.138736 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.139033 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.139124 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:15.139281 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:15.139463 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.129886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44699 (local address 127.29.145.2:41107)
0811 11:10:15.130066 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.130115 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.131096 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.131427 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.131432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.131710 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:15.131906 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.131911 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.132627 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.132628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.133468 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.133471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.133935 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.133936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.134071 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.134716 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:15.134725 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:15.136197 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:15.137846 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:15.137854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:15.137857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:15.138105 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:15.138341 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:15.138342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:15.138343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:15.138453 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:15.138744 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:15.138751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:15.138891 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.138999 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.139168 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260811 11:10:15.139492 30276 external_mini_cluster.cc:1428] Setting key fbe76a44e765e8efdfc0130002993051
I20260811 11:10:15.139978 31262 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44699
I20260811 11:10:15.140100 31262 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:15.140305 31262 heartbeater.cc:511] Master 127.29.145.62:44699 requested a full tablet report, sending...
I20260811 11:10:15.140748 30900 ts_manager.cc:194] Registered new tserver with Master: 30edf72f60604dc4965e0e4010726e4f (127.29.145.2:34309)
I20260811 11:10:15.141126 30900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:41107
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:15.169339 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:15.259228 31269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:15.259456 31269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:15.259481 31269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:15.260605 31269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:15.260648 31269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:15.260674 31269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:15.262614 31269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:15.262884 31269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:15.263129 31269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:15.268294 31275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:15.268302 31276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:15.268471 31269 server_base.cc:1034] running on GCE node
W20260811 11:10:15.268323 31278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:15.268867 31269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:15.269132 31269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:15.270323 31269 hybrid_clock.cc:648] HybridClock initialized: now 1786446615270316 us; error 39 us; skew 500 ppm
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:15.272957 31269 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:15.273409 31269 webserver.cc:460] Webserver started at http://127.29.145.3:42237/ using document root <none> and password file <none>
I20260811 11:10:15.273658 31269 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:15.273731 31269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:15.274091 31269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:15.275413 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "4d26a043769f49c7a0d909afd31b0342"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "37c926f136d09bc96c18000a6d329802"
server_key_iv: "21fc685cd22f7fd1dba17a558138c5e9"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.275833 31269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "4d26a043769f49c7a0d909afd31b0342"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "37c926f136d09bc96c18000a6d329802"
server_key_iv: "21fc685cd22f7fd1dba17a558138c5e9"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.277486 31269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:15.278429 31285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.278702 31269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:15.278839 31269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "4d26a043769f49c7a0d909afd31b0342"
format_stamp: "Formatted at 2026-08-11 11:10:15 on dist-test-slave-69rg"
server_key: "37c926f136d09bc96c18000a6d329802"
server_key_iv: "21fc685cd22f7fd1dba17a558138c5e9"
server_key_version: "encryptionkey@0"
I20260811 11:10:15.278923 31269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:15.298223 31269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:15.300671 31269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:15.300866 31269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:15.301138 31269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:15.301482 31269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:15.301539 31269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.301584 31269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:15.301615 31269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:15.310900 31269 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:40651
I20260811 11:10:15.310942 31408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:40651 every 8 connection(s)
I20260811 11:10:15.311328 31269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:15.315356 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31269
I20260811 11:10:15.315502 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:15.315693 30276 external_mini_cluster.cc:1428] Setting key 1de30cdb1cfab1e346322a204718b228
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:15.322738 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.312259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:47167 (local address 127.29.145.62:44699)
0811 11:10:15.312353 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.312355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.312632 (+ 277us) server_negotiation.cc:407] Connection header received
0811 11:10:15.313702 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.313703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.313715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.313785 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.314741 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.315279 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.316687 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.316978 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.317057 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.319751 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.319753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.319753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.319762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.321129 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.321613 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.321614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.321615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.321689 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.322010 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.322011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.322012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.322159 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.322243 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.322545 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.322639 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:10:15.322976 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.312239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44699 (local address 127.29.145.3:47167)
0811 11:10:15.312456 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.312505 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.313551 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.313935 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.313940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.314266 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:15.314489 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.314494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.315425 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.315426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.316606 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.316610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.317064 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.317066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.317212 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.317940 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:15.317951 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:15.319609 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:15.321261 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:15.321270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:15.321273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:15.321522 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:15.321762 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:15.321763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:15.321764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:15.321936 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:15.322231 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:15.322238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:15.322394 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.322503 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.322692 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":122,"threads_started":1}
I20260811 11:10:15.323568 31410 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44699
I20260811 11:10:15.323670 31410 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:15.323830 31410 heartbeater.cc:511] Master 127.29.145.62:44699 requested a full tablet report, sending...
I20260811 11:10:15.324286 30900 ts_manager.cc:194] Registered new tserver with Master: 4d26a043769f49c7a0d909afd31b0342 (127.29.145.3:40651)
I20260811 11:10:15.324681 30900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:47167
I20260811 11:10:15.328208 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:10:15.337038 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.330086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.29.145.62:44699)
0811 11:10:15.330150 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.330152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.330275 (+ 123us) server_negotiation.cc:407] Connection header received
0811 11:10:15.330330 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.330331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.330340 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.330395 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.331055 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.331544 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.332323 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.332550 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.332639 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.334052 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.334054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.334054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.334061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.335623 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.336014 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.336015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.336016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.336071 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.336341 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.336342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.336343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.336524 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.336624 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.336870 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.336961 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:15.339839 30900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:15.341604 30900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:15.348942 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.29.145.1:45673)
0811 11:10:15.344947 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.344953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.345121 (+ 168us) server_negotiation.cc:407] Connection header received
0811 11:10:15.345178 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.345180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.345197 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.345448 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:15.345539 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346670 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.347954 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348461 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348576 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348644 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.348814 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.348857 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
I20260811 11:10:15.348927 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:40651 (local address 127.0.0.1:39234)
0811 11:10:15.344740 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.344785 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.344864 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.345212 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.345217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.345219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:15.345301 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.345303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346446 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.346447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.347693 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.347696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348621 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.348704 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348709 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.348740 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.348806 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":65,"threads_started":1}
I20260811 11:10:15.349128 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42650 (local address 127.29.145.2:34309)
0811 11:10:15.345002 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.345006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.345282 (+ 276us) server_negotiation.cc:407] Connection header received
0811 11:10:15.345329 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.345331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.345348 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.345421 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:15.345638 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346752 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.347943 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348628 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348689 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348746 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.348978 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.349023 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260811 11:10:15.349179 31422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:45673 (local address 127.0.0.1:34418)
0811 11:10:15.345004 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.345014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.345087 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.345302 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.345303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.345305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:15.345368 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.345369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346782 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.346783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.347863 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.347865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348579 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.348665 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348680 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.348793 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.348839 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":39,"spinlock_wait_cycles":768,"thread_start_us":121,"threads_started":1}
I20260811 11:10:15.349278 31421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34309 (local address 127.0.0.1:42650)
0811 11:10:15.345160 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.345169 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.345221 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.345513 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.345514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.345516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:15.345551 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.345552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346833 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.346834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.347824 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.347827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348780 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.348829 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348833 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.348881 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.348918 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":3,"thread_start_us":142,"threads_started":1}
I20260811 11:10:15.349599 31426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.344742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.29.145.3:40651)
0811 11:10:15.344947 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.344953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.344965 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:15.345011 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.345014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.345034 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.345385 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:15.345476 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.346333 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.347790 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.348511 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.348891 (+ 380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.348952 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.348977 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.349170 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260811 11:10:15.350103 31036 tablet_service.cc:1467] Processing CreateTablet for tablet 58a9339d22724759a6829b987b4799b3 (DEFAULT_TABLE table=test-table [id=66fd5996291c4385b2671b6b8b79bcc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:15.350092 31187 tablet_service.cc:1467] Processing CreateTablet for tablet 58a9339d22724759a6829b987b4799b3 (DEFAULT_TABLE table=test-table [id=66fd5996291c4385b2671b6b8b79bcc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:15.350338 31335 tablet_service.cc:1467] Processing CreateTablet for tablet 58a9339d22724759a6829b987b4799b3 (DEFAULT_TABLE table=test-table [id=66fd5996291c4385b2671b6b8b79bcc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:15.350474 31187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a9339d22724759a6829b987b4799b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:15.350464 31036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a9339d22724759a6829b987b4799b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:15.350632 31335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58a9339d22724759a6829b987b4799b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:15.353199 31428 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Bootstrap starting.
I20260811 11:10:15.353981 31428 tablet_bootstrap.cc:654] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:15.354103 31430 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Bootstrap starting.
I20260811 11:10:15.354167 31429 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Bootstrap starting.
I20260811 11:10:15.354579 31428 log.cc:824] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:15.354800 31430 tablet_bootstrap.cc:654] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:15.354800 31429 tablet_bootstrap.cc:654] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:15.355379 31428 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: No bootstrap required, opened a new log
I20260811 11:10:15.355423 31430 log.cc:824] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:15.355423 31429 log.cc:824] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:15.355473 31428 ts_tablet_manager.cc:1397] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 11:10:15.356187 31430 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: No bootstrap required, opened a new log
I20260811 11:10:15.356263 31430 ts_tablet_manager.cc:1397] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:15.356444 31429 tablet_bootstrap.cc:492] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: No bootstrap required, opened a new log
I20260811 11:10:15.356514 31429 ts_tablet_manager.cc:1397] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:15.357654 31428 raft_consensus.cc:348] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.357893 31428 raft_consensus.cc:374] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:15.357957 31428 raft_consensus.cc:729] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793e9d1055f04fdebb3f328725464861, State: Initialized, Role: FOLLOWER
I20260811 11:10:15.358107 31428 consensus_queue.cc:260] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.358433 31108 heartbeater.cc:503] Master 127.29.145.62:44699 was elected leader, sending a full tablet report...
I20260811 11:10:15.358443 31428 ts_tablet_manager.cc:1428] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:10:15.358384 31430 raft_consensus.cc:348] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.358551 31430 raft_consensus.cc:374] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:15.358592 31430 raft_consensus.cc:729] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d26a043769f49c7a0d909afd31b0342, State: Initialized, Role: FOLLOWER
I20260811 11:10:15.358553 31429 raft_consensus.cc:348] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.358711 31429 raft_consensus.cc:374] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:15.358759 31429 raft_consensus.cc:729] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30edf72f60604dc4965e0e4010726e4f, State: Initialized, Role: FOLLOWER
I20260811 11:10:15.358722 31430 consensus_queue.cc:260] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.358867 31429 consensus_queue.cc:260] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.359019 31430 ts_tablet_manager.cc:1428] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:10:15.359022 31410 heartbeater.cc:503] Master 127.29.145.62:44699 was elected leader, sending a full tablet report...
I20260811 11:10:15.359197 31429 ts_tablet_manager.cc:1428] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:15.359318 31262 heartbeater.cc:503] Master 127.29.145.62:44699 was elected leader, sending a full tablet report...
W20260811 11:10:15.379720 31263 tablet.cc:2369] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:15.457865 31109 tablet.cc:2369] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:15.479729 31435 raft_consensus.cc:482] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:15.479871 31435 raft_consensus.cc:504] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.480284 31435 leader_election.cc:288] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 793e9d1055f04fdebb3f328725464861 (127.29.145.1:45673), 30edf72f60604dc4965e0e4010726e4f (127.29.145.2:34309)
I20260811 11:10:15.484798 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.480585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:49261 (local address 127.29.145.1:45673)
0811 11:10:15.480655 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.480657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.480815 (+ 158us) server_negotiation.cc:407] Connection header received
0811 11:10:15.480849 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.480850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.480863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.480916 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:15.481175 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.481979 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.483716 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.484357 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.484425 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.484433 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.484680 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.484726 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:10:15.484798 31427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.480691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:39043 (local address 127.29.145.2:34309)
0811 11:10:15.480741 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.480743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.480751 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:15.480808 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.480809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.480827 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.480889 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:15.481167 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.481978 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.483725 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.484312 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.484398 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.484410 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.484688 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.484735 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:10:15.484789 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.480516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34309 (local address 127.29.145.3:39043)
0811 11:10:15.480596 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.480608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.480687 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.480930 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.480931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.480933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:15.481055 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.481057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.482079 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.482081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.483597 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.483600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.484444 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.484445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.484550 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.484552 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.484624 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.484674 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":123}
I20260811 11:10:15.484792 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.480538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:45673 (local address 127.29.145.3:49261)
0811 11:10:15.480726 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:15.480745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:15.480802 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:15.480930 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:15.480931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:15.480933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:15.481055 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.481057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.482079 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.482081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.483597 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:15.483600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.484444 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.484445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:15.484545 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.484551 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:10:15.484624 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 11:10:15.484673 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":78,"thread_start_us":136,"threads_started":1}
I20260811 11:10:15.485200 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a9339d22724759a6829b987b4799b3" candidate_uuid: "4d26a043769f49c7a0d909afd31b0342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793e9d1055f04fdebb3f328725464861" is_pre_election: true
I20260811 11:10:15.485224 31211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a9339d22724759a6829b987b4799b3" candidate_uuid: "4d26a043769f49c7a0d909afd31b0342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30edf72f60604dc4965e0e4010726e4f" is_pre_election: true
I20260811 11:10:15.485401 31059 raft_consensus.cc:2454] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d26a043769f49c7a0d909afd31b0342 in term 0.
I20260811 11:10:15.485412 31211 raft_consensus.cc:2454] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d26a043769f49c7a0d909afd31b0342 in term 0.
I20260811 11:10:15.485680 31286 leader_election.cc:302] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d26a043769f49c7a0d909afd31b0342, 793e9d1055f04fdebb3f328725464861; no voters:
I20260811 11:10:15.485862 31435 raft_consensus.cc:2781] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:15.485926 31435 raft_consensus.cc:482] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:15.485965 31435 raft_consensus.cc:3037] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:15.486771 31435 raft_consensus.cc:504] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.486946 31435 leader_election.cc:288] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [CANDIDATE]: Term 1 election: Requested vote from peers 793e9d1055f04fdebb3f328725464861 (127.29.145.1:45673), 30edf72f60604dc4965e0e4010726e4f (127.29.145.2:34309)
I20260811 11:10:15.487141 31211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a9339d22724759a6829b987b4799b3" candidate_uuid: "4d26a043769f49c7a0d909afd31b0342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30edf72f60604dc4965e0e4010726e4f"
I20260811 11:10:15.487244 31211 raft_consensus.cc:3037] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:15.487532 31059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58a9339d22724759a6829b987b4799b3" candidate_uuid: "4d26a043769f49c7a0d909afd31b0342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793e9d1055f04fdebb3f328725464861"
I20260811 11:10:15.487707 31059 raft_consensus.cc:3037] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:15.488073 31211 raft_consensus.cc:2454] T 58a9339d22724759a6829b987b4799b3 P 30edf72f60604dc4965e0e4010726e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d26a043769f49c7a0d909afd31b0342 in term 1.
I20260811 11:10:15.488245 31289 leader_election.cc:302] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30edf72f60604dc4965e0e4010726e4f, 4d26a043769f49c7a0d909afd31b0342; no voters:
I20260811 11:10:15.488346 31435 raft_consensus.cc:2781] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:15.488557 31435 raft_consensus.cc:686] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [term 1 LEADER]: Becoming Leader. State: Replica: 4d26a043769f49c7a0d909afd31b0342, State: Running, Role: LEADER
I20260811 11:10:15.488677 31435 consensus_queue.cc:237] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } }
I20260811 11:10:15.488729 31059 raft_consensus.cc:2454] T 58a9339d22724759a6829b987b4799b3 P 793e9d1055f04fdebb3f328725464861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d26a043769f49c7a0d909afd31b0342 in term 1.
I20260811 11:10:15.489739 30900 catalog_manager.cc:5495] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d26a043769f49c7a0d909afd31b0342 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "4d26a043769f49c7a0d909afd31b0342" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "793e9d1055f04fdebb3f328725464861" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d26a043769f49c7a0d909afd31b0342" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 40651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30edf72f60604dc4965e0e4010726e4f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34309 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:15.501223 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.492756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34430 (local address 127.29.145.1:45673)
0811 11:10:15.492923 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.492925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.492934 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:15.492985 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.492985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.492995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.493051 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.493614 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.494190 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.494965 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.495190 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.495256 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.498131 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.498151 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.498151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.498158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.499913 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.500330 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.500337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.500337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.500408 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.500622 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.500623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.500623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.500787 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.500945 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.501023 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.501112 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:15.532402 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.521239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34444 (local address 127.29.145.1:45673)
0811 11:10:15.521377 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.521379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.521389 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:15.521418 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.521419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.521429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.521492 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.522845 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.523713 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.524796 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.525010 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.525084 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.528266 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.528268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.528268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.528275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.530373 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.531213 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.531214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.531214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.531282 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.531479 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.531480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.531480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.531584 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.531703 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.532007 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.532124 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[30848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:15.560973 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:15.551687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.29.145.1:45673)
0811 11:10:15.551831 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:15.551833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:15.551931 (+ 98us) server_negotiation.cc:407] Connection header received
0811 11:10:15.552050 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:15.552051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:15.552064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:15.552115 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:15.552733 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.553220 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.554025 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:15.554223 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:15.554299 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:15.557336 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:15.557338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:15.557338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:15.557346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:15.559239 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.559633 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.559634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.559635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.559706 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:15.560224 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:15.560226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:15.560226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:15.560428 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:15.560573 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:15.560821 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 11:10:15.560896 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 11:10:15.561599 31014 tablet_service.cc:2908] Rejecting scan request for tablet 58a9339d22724759a6829b987b4799b3: Uninitialized: clean time has not yet been initialized
W20260811 11:10:15.562023 31411 tablet.cc:2369] T 58a9339d22724759a6829b987b4799b3 P 4d26a043769f49c7a0d909afd31b0342: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:15.562608 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30966
I20260811 11:10:15.571311 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31113
I20260811 11:10:15.582976 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31269
I20260811 11:10:15.590838 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 30862
2026-08-11T11:10:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2238 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[31454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[31454](info): set up 2 sockets
Aug 11 11:10:15 dist-test-slave-69rg krb5kdc[31454](info): commencing operation
krb5kdc: starting...
W20260811 11:10:16.785372 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:16 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:16.941697 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37463
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:17.046096 31469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:17.046383 31469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:17.046419 31469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:17.049064 31469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:17.049172 31469 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:10:17.049197 31469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:17.049217 31469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:17.049234 31469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:17.049252 31469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:17.051678 31469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37463
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:17.052003 31469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:17.052275 31469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:17.058622 31475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.058816 31469 server_base.cc:1034] running on GCE node
W20260811 11:10:17.058676 31478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.058622 31476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.059350 31469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:17.059702 31469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:17.060915 31469 hybrid_clock.cc:648] HybridClock initialized: now 1786446617060895 us; error 33 us; skew 500 ppm
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:17.064260 31469 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:17.064903 31469 webserver.cc:460] Webserver started at http://127.29.145.62:40215/ using document root <none> and password file <none>
I20260811 11:10:17.065194 31469 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:17.065272 31469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:17.065449 31469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:17.066883 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "9d439cefb8054acaa152011843eaeebd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "a2b909305a4bb454705faa1a212a9b21"
server_key_iv: "1ddb82f5baee62fa3ffa9b7c13677b7a"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.067334 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "9d439cefb8054acaa152011843eaeebd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "a2b909305a4bb454705faa1a212a9b21"
server_key_iv: "1ddb82f5baee62fa3ffa9b7c13677b7a"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.069054 31469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:17.070044 31485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.070253 31469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:17.070372 31469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "9d439cefb8054acaa152011843eaeebd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "a2b909305a4bb454705faa1a212a9b21"
server_key_iv: "1ddb82f5baee62fa3ffa9b7c13677b7a"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.070454 31469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:17.079672 31469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:17.083222 31469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:17.083467 31469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:17.090952 31469 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37463
I20260811 11:10:17.091141 31561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37463 every 8 connection(s)
I20260811 11:10:17.091523 31469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:17.092664 31562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:17.096032 31562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Bootstrap starting.
I20260811 11:10:17.096926 31562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:17.097337 31562 log.cc:824] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:17.098771 31562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: No bootstrap required, opened a new log
I20260811 11:10:17.100337 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31469
I20260811 11:10:17.100445 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:17.100670 30276 external_mini_cluster.cc:1428] Setting key 8893231a70619e7e5a7580300b00b10b
I20260811 11:10:17.102705 31562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } }
I20260811 11:10:17.103000 31562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:17.103051 31562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d439cefb8054acaa152011843eaeebd, State: Initialized, Role: FOLLOWER
I20260811 11:10:17.103253 31562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } }
I20260811 11:10:17.103389 31562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:17.103458 31562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:17.103513 31562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:17.104534 31562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } }
I20260811 11:10:17.104679 31562 leader_election.cc:302] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d439cefb8054acaa152011843eaeebd; no voters:
I20260811 11:10:17.104876 31562 leader_election.cc:288] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:17.104955 31568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:17.105141 31568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [term 1 LEADER]: Becoming Leader. State: Replica: 9d439cefb8054acaa152011843eaeebd, State: Running, Role: LEADER
I20260811 11:10:17.105257 31562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:17.105304 31568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } }
I20260811 11:10:17.105859 31569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d439cefb8054acaa152011843eaeebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } } }
I20260811 11:10:17.105957 31569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:17.106231 31570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d439cefb8054acaa152011843eaeebd. Latest consensus state: current_term: 1 leader_uuid: "9d439cefb8054acaa152011843eaeebd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d439cefb8054acaa152011843eaeebd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37463 } } }
I20260811 11:10:17.106348 31570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:17.107031 31575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:17.108006 31575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:17.109961 31575 catalog_manager.cc:1347] Generated new cluster ID: cfd72931187f47fdb86245dd01e5bb7f
I20260811 11:10:17.110034 31575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:17.116428 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.101465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.29.145.62:37463)
0811 11:10:17.101853 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.101870 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.101937 (+ 67us) server_negotiation.cc:407] Connection header received
0811 11:10:17.102555 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.102567 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.102756 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.102998 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.103988 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.105520 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.106811 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.107526 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.107653 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.111532 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.111540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.111541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.111564 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.114038 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.114746 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.114751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.114752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.114934 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.115212 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.115213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.115213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.115589 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.115798 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.115904 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.116142 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":141,"threads_started":1}
I20260811 11:10:17.116772 31575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:17.117543 31575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:17.127429 31575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Generated new TSK 0
I20260811 11:10:17.127766 31575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:17.161590 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37463
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:17.269274 31591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:17.269590 31591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:17.269640 31591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:17.271056 31591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:17.271137 31591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:17.271175 31591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:17.271217 31591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:17.272930 31591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:17.273273 31591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:17.273617 31591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:17.280303 31598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.280303 31597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.280296 31600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.281554 31591 server_base.cc:1034] running on GCE node
I20260811 11:10:17.281881 31591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:17.282276 31591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:17.283495 31591 hybrid_clock.cc:648] HybridClock initialized: now 1786446617283468 us; error 44 us; skew 500 ppm
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:17.287628 31591 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:17.288261 31591 webserver.cc:460] Webserver started at http://127.29.145.1:40387/ using document root <none> and password file <none>
I20260811 11:10:17.288599 31591 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:17.288667 31591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:17.288836 31591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:17.290349 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "c160d87cb8b040bb85b9fd6378dd40e4"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "4e8b2153f311d510231ea39ced2dcd36"
server_key_iv: "df63e1044b47f0459cd36a60cb4b2e81"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.290889 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "c160d87cb8b040bb85b9fd6378dd40e4"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "4e8b2153f311d510231ea39ced2dcd36"
server_key_iv: "df63e1044b47f0459cd36a60cb4b2e81"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.292768 31591 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:17.293951 31607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.294147 31591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:17.294272 31591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "c160d87cb8b040bb85b9fd6378dd40e4"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "4e8b2153f311d510231ea39ced2dcd36"
server_key_iv: "df63e1044b47f0459cd36a60cb4b2e81"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.294359 31591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:17.339241 31591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:17.342618 31591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:17.342778 31591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:17.343400 31591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:17.343439 31591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.343466 31591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:17.343478 31591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.351680 31591 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:34515
I20260811 11:10:17.351774 31737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:34515 every 8 connection(s)
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:17.352190 31591 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:17.356019 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.344203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45869 (local address 127.29.145.62:37463)
0811 11:10:17.344296 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.344298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.344565 (+ 267us) server_negotiation.cc:407] Connection header received
0811 11:10:17.345309 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.345311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.345336 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.345428 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.346307 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.347256 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.348981 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.349351 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.349460 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.352731 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.352734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.352734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.352743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.354336 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.354802 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.354803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.354804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.354869 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.355208 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.355209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.355210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.355355 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.355423 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.355820 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.355924 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:10:17.356258 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.344166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.1:45869)
0811 11:10:17.344378 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.344427 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.345154 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.345541 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.345548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.345978 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.346223 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.346229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.347376 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.347377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.348893 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.348896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.349466 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.349468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.349628 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.350285 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.350302 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.352603 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.354455 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.354463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.354466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.354716 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.354966 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.354967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.354968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.355101 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.355428 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.355435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.355625 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.355756 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.355930 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260811 11:10:17.357093 31738 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37463
I20260811 11:10:17.357235 31738 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:17.357524 31738 heartbeater.cc:511] Master 127.29.145.62:37463 requested a full tablet report, sending...
I20260811 11:10:17.358258 31506 ts_manager.cc:194] Registered new tserver with Master: c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515)
I20260811 11:10:17.359048 31506 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:45869
I20260811 11:10:17.360893 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31591
I20260811 11:10:17.360991 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:17.361208 30276 external_mini_cluster.cc:1428] Setting key 64a10b79d93bff3a093489b6c707e71c
I20260811 11:10:17.365550 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.358575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.1:58315)
0811 11:10:17.358732 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.358743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.358837 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.358992 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.358993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.359273 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.359386 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.359389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.360297 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.360298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.361016 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.361019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.361507 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.361508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.361621 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.362167 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.362172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.362534 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.364375 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.364376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.364376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.364562 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.364818 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.364819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.364819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.364900 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.365238 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.365239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.365285 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.365345 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.365461 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260811 11:10:17.365648 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.358631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:58315 (local address 127.29.145.62:37463)
0811 11:10:17.358683 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.358686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.358819 (+ 133us) server_negotiation.cc:407] Connection header received
0811 11:10:17.358895 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.358896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.358909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.358988 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.359471 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.360200 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.361106 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.361385 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.361469 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.362637 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.362638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.362639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.362647 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.364276 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.364644 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.364646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.364646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.364724 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.364990 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.364991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.364992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.365163 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.365247 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.365386 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.365493 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:17.394035 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37463
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:17.487701 31749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:17.487907 31749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:17.487931 31749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:17.489144 31749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:17.489187 31749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:17.489207 31749 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:17.489261 31749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:17.490794 31749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:17.491067 31749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:17.491321 31749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:17.496570 31754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.496549 31755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.496542 31757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.496870 31749 server_base.cc:1034] running on GCE node
I20260811 11:10:17.497097 31749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:17.497342 31749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:17.498533 31749 hybrid_clock.cc:648] HybridClock initialized: now 1786446617498521 us; error 39 us; skew 500 ppm
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:17.501272 31749 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:17.501775 31749 webserver.cc:460] Webserver started at http://127.29.145.2:35335/ using document root <none> and password file <none>
I20260811 11:10:17.502054 31749 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:17.502102 31749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:17.502198 31749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:17.503244 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "fa05721a3c9342f0869a6c824936d7fd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "e2c7a67743eee802e642bae531a57acd"
server_key_iv: "cccf59455acad447a180d2b7146fafbd"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.503556 31749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "fa05721a3c9342f0869a6c824936d7fd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "e2c7a67743eee802e642bae531a57acd"
server_key_iv: "cccf59455acad447a180d2b7146fafbd"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.504879 31749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:17.505852 31764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.506080 31749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:17.506214 31749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "fa05721a3c9342f0869a6c824936d7fd"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "e2c7a67743eee802e642bae531a57acd"
server_key_iv: "cccf59455acad447a180d2b7146fafbd"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.506294 31749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:17.527690 31749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:17.529765 31749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:17.529949 31749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:17.530544 31749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:17.530587 31749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.530612 31749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:17.530625 31749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:17.538014 31749 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:41897
I20260811 11:10:17.538156 31897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:41897 every 8 connection(s)
I20260811 11:10:17.538473 31749 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:17.540607 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.531062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:34939 (local address 127.29.145.62:37463)
0811 11:10:17.531145 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.531148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.531369 (+ 221us) server_negotiation.cc:407] Connection header received
0811 11:10:17.531846 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.531847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.531862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.531942 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.532848 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.533705 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.534758 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.535091 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.535176 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.537682 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.537683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.537684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.537692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.539298 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.539765 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.539766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.539766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.539822 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.540022 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.540022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.540023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.540126 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.540183 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.540385 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.540445 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:10:17.540660 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31749
I20260811 11:10:17.540705 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.530987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.2:34939)
0811 11:10:17.531214 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.531256 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.531742 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.532011 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.532015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.532436 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.532736 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.532744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.533839 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.533840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.534661 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.534664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.535211 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.535212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.535384 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.535903 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.535912 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.537579 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.539375 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.539380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.539382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.539689 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.539886 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.539886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.539887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.539982 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.540179 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.540184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.540291 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.540369 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.540485 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260811 11:10:17.540746 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:17.541090 30276 external_mini_cluster.cc:1428] Setting key c8ed8c5d69c4c228cc6890cf1b8f50e7
I20260811 11:10:17.541266 31898 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37463
I20260811 11:10:17.541368 31898 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:17.541563 31898 heartbeater.cc:511] Master 127.29.145.62:37463 requested a full tablet report, sending...
I20260811 11:10:17.541960 31505 ts_manager.cc:194] Registered new tserver with Master: fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897)
I20260811 11:10:17.542438 31505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:34939
I20260811 11:10:17.549777 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.542443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.2:37605)
0811 11:10:17.543898 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.543909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.543965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.544113 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.544114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.544298 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.544346 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.544347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.545281 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.545282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.545795 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.545796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.546228 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.546229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.546302 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.546772 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.546776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.547042 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.548677 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.548678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.548679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.548854 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.549071 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.549072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.549072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.549138 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.549434 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.549436 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.549481 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.549560 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.549676 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"thread_start_us":139,"threads_started":1}
I20260811 11:10:17.549777 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.542506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:37605 (local address 127.29.145.62:37463)
0811 11:10:17.542541 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.542543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.543988 (+ 1445us) server_negotiation.cc:407] Connection header received
0811 11:10:17.544017 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.544018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.544030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.544103 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.544421 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.545161 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.545935 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.546139 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.546224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.547135 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.547136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.547136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.547143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.548572 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.548930 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.548931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.548931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.548990 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.549208 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.549209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.549210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.549357 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.549430 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.549602 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.549674 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:17.573388 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37463
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:17.666137 31909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:17.666337 31909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:17.666360 31909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:17.667498 31909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:17.667550 31909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:17.667570 31909 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:17.667615 31909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:17.669045 31909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:37463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:17.669286 31909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:17.669520 31909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:17.674717 31917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:17.674695 31914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.675004 31909 server_base.cc:1034] running on GCE node
W20260811 11:10:17.674695 31915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:17.675237 31909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:17.675482 31909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:17.676631 31909 hybrid_clock.cc:648] HybridClock initialized: now 1786446617676612 us; error 40 us; skew 500 ppm
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:17.679388 31909 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:17.679859 31909 webserver.cc:460] Webserver started at http://127.29.145.3:34345/ using document root <none> and password file <none>
I20260811 11:10:17.680089 31909 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:17.680135 31909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:17.680272 31909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:17.681403 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "e839036714e84ea2bb80d7a3e40e841f"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "d28677fa72a93fade0389b72747866ce"
server_key_iv: "3ef800c7ae147bb814d11ab7e928b3fa"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.681793 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "e839036714e84ea2bb80d7a3e40e841f"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "d28677fa72a93fade0389b72747866ce"
server_key_iv: "3ef800c7ae147bb814d11ab7e928b3fa"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.683307 31909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:17.684247 31925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.684477 31909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:17.684580 31909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "e839036714e84ea2bb80d7a3e40e841f"
format_stamp: "Formatted at 2026-08-11 11:10:17 on dist-test-slave-69rg"
server_key: "d28677fa72a93fade0389b72747866ce"
server_key_iv: "3ef800c7ae147bb814d11ab7e928b3fa"
server_key_version: "encryptionkey@0"
I20260811 11:10:17.684664 31909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:17.695930 31909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:17.698567 31909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:17.698724 31909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:17.699342 31909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:17.699397 31909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.699437 31909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:17.699477 31909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:17.707100 31909 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:38109
I20260811 11:10:17.707230 32054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:38109 every 8 connection(s)
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:17.707623 31909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:17.709609 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 31909
I20260811 11:10:17.709709 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:17.709926 30276 external_mini_cluster.cc:1428] Setting key f8ac5dd058831587ca12b1585e524ce4
I20260811 11:10:17.711259 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.699942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:39821 (local address 127.29.145.62:37463)
0811 11:10:17.700021 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.700024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.700260 (+ 236us) server_negotiation.cc:407] Connection header received
0811 11:10:17.701070 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.701072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.701137 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.701213 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.702164 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.702666 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.704377 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.704597 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.704686 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.707953 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.707956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.707956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.707966 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.709519 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.710209 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.710211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.710211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.710289 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.710570 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.710579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.710580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.710704 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.710783 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.711082 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.711168 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:10:17.711452 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.699917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.3:39821)
0811 11:10:17.700100 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.700153 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.700899 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.701304 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.701311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.701718 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.702053 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.702060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.702775 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.702776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.704271 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.704275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.704709 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.704711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.704845 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.705983 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.706001 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.707851 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.709634 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.709640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.709642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.710043 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.710369 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.710370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.710370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.710492 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.710788 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.710795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.710938 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.711039 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.711217 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260811 11:10:17.711978 32055 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37463
I20260811 11:10:17.712067 32055 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:17.712306 32055 heartbeater.cc:511] Master 127.29.145.62:37463 requested a full tablet report, sending...
I20260811 11:10:17.712693 31505 ts_manager.cc:194] Registered new tserver with Master: e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109)
I20260811 11:10:17.713272 31505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:39821
I20260811 11:10:17.718676 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.713173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:34633 (local address 127.29.145.62:37463)
0811 11:10:17.713221 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.713223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.713300 (+ 77us) server_negotiation.cc:407] Connection header received
0811 11:10:17.713393 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.713394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.713404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.713468 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.713841 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.714336 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.715037 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.715243 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.715318 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.716217 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.716218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.716218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.716226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.717466 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.717855 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.717856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.717857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.717919 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.718138 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.718139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.718139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.718262 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.718342 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.718504 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.718585 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:17.718674 32062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.713103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.29.145.3:34633)
0811 11:10:17.713235 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.713247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.713307 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.713494 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.713495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.713677 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:17.713726 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.713728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.714435 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.714436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.714949 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.714951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.715330 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.715331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.715411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.715804 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:17.715809 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:17.716123 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:17.717574 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.717575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.717576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.717761 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.717995 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:17.717996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:17.717997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:17.718062 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:17.718334 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:17.718335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:17.718382 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.718470 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.718575 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260811 11:10:17.721522 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:10:17 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:17.745966 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.738424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47832 (local address 127.29.145.62:37463)
0811 11:10:17.738498 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.738500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.738508 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:17.738540 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.738541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.738550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.738632 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:17.739245 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.739743 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.740536 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.740739 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.740816 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.743393 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:17.743395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:17.743395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:17.743402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:17.744711 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.745126 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.745127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.745128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.745183 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:17.745402 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:17.745403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:17.745403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:17.745522 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:17.745609 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.745771 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.745900 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:10:17.747685 31505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:17.749432 31505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:17.756563 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.29.145.1:34515)
0811 11:10:17.752383 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.752390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.752403 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:10:17.752456 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.752459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.752478 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.752570 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:17.752839 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754057 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.755391 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756069 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.756149 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.756209 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.756388 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.756431 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260811 11:10:17.756511 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:34515 (local address 127.0.0.1:51594)
0811 11:10:17.752209 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.752254 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.752323 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.752588 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.752592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.752659 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:17.752737 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.752739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754171 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.754172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.755290 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.755293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756169 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.756170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.756259 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.756274 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.756359 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.756420 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":65,"thread_start_us":67,"threads_started":1}
I20260811 11:10:17.757089 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:41897 (local address 127.0.0.1:54444)
0811 11:10:17.752950 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.752960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.753008 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.753218 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.753219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.753221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:17.753259 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.753261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754468 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.754469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.755751 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.755753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756840 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.756841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.756907 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.756911 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.756969 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.757017 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260811 11:10:17.757331 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:38109 (local address 127.0.0.1:37850)
0811 11:10:17.752411 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:17.752424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:17.752475 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:17.752896 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:17.752897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:17.752899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:17.752955 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.752957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754114 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.754115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.755630 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:17.755633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756583 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.756585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:17.757024 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.757029 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:17.757067 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 11:10:17.757130 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":40,"thread_start_us":96,"threads_started":1}
I20260811 11:10:17.757436 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.29.145.2:41897)
0811 11:10:17.752653 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.752657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.753064 (+ 407us) server_negotiation.cc:407] Connection header received
0811 11:10:17.753108 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.753110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.753125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.753201 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:17.753343 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754380 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.755843 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756634 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.756707 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.756796 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.757286 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.757329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260811 11:10:17.757452 31666 tablet_service.cc:1467] Processing CreateTablet for tablet 06a65d76c3b544969fb7671409254bd5 (DEFAULT_TABLE table=test-table [id=43452ade89254be5be328d0f2a11117b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:17.757781 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:17.752198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37850 (local address 127.29.145.3:38109)
0811 11:10:17.752703 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:17.752707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:17.752718 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:17.752756 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:17.752758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:17.752775 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:17.752854 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:17.753050 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.754025 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.755735 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:17.756476 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:17.757533 (+ 1057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:17.757608 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:17.757635 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 11:10:17.757676 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":94,"threads_started":1}
I20260811 11:10:17.758010 31666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06a65d76c3b544969fb7671409254bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:17.758150 31823 tablet_service.cc:1467] Processing CreateTablet for tablet 06a65d76c3b544969fb7671409254bd5 (DEFAULT_TABLE table=test-table [id=43452ade89254be5be328d0f2a11117b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:17.758496 31823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06a65d76c3b544969fb7671409254bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:17.762236 31977 tablet_service.cc:1467] Processing CreateTablet for tablet 06a65d76c3b544969fb7671409254bd5 (DEFAULT_TABLE table=test-table [id=43452ade89254be5be328d0f2a11117b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:17.762548 32080 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Bootstrap starting.
I20260811 11:10:17.762619 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06a65d76c3b544969fb7671409254bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:17.763314 32080 tablet_bootstrap.cc:654] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:17.763674 32080 log.cc:824] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:17.764055 32079 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Bootstrap starting.
I20260811 11:10:17.764464 32080 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: No bootstrap required, opened a new log
I20260811 11:10:17.764516 32080 ts_tablet_manager.cc:1397] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:17.764736 32079 tablet_bootstrap.cc:654] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:17.765097 32079 log.cc:824] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:17.766021 32079 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: No bootstrap required, opened a new log
I20260811 11:10:17.766086 32079 ts_tablet_manager.cc:1397] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:17.766417 32083 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Bootstrap starting.
I20260811 11:10:17.766309 32080 raft_consensus.cc:348] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.766494 32080 raft_consensus.cc:374] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:17.766532 32080 raft_consensus.cc:729] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c160d87cb8b040bb85b9fd6378dd40e4, State: Initialized, Role: FOLLOWER
I20260811 11:10:17.766680 32080 consensus_queue.cc:260] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.767043 32080 ts_tablet_manager.cc:1428] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:10:17.767228 31738 heartbeater.cc:503] Master 127.29.145.62:37463 was elected leader, sending a full tablet report...
I20260811 11:10:17.767349 32083 tablet_bootstrap.cc:654] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:17.767719 32083 log.cc:824] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:17.768518 32083 tablet_bootstrap.cc:492] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: No bootstrap required, opened a new log
I20260811 11:10:17.768432 32079 raft_consensus.cc:348] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.768576 32083 ts_tablet_manager.cc:1397] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:17.768582 32079 raft_consensus.cc:374] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:17.768640 32079 raft_consensus.cc:729] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa05721a3c9342f0869a6c824936d7fd, State: Initialized, Role: FOLLOWER
I20260811 11:10:17.768761 32079 consensus_queue.cc:260] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.769042 32079 ts_tablet_manager.cc:1428] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:17.769119 31898 heartbeater.cc:503] Master 127.29.145.62:37463 was elected leader, sending a full tablet report...
I20260811 11:10:17.770624 32083 raft_consensus.cc:348] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.770777 32083 raft_consensus.cc:374] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:17.770828 32083 raft_consensus.cc:729] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e839036714e84ea2bb80d7a3e40e841f, State: Initialized, Role: FOLLOWER
I20260811 11:10:17.770954 32083 consensus_queue.cc:260] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:17.771227 32083 ts_tablet_manager.cc:1428] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:17.771286 32055 heartbeater.cc:503] Master 127.29.145.62:37463 was elected leader, sending a full tablet report...
W20260811 11:10:17.789742 31899 tablet.cc:2369] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:17.853080 31739 tablet.cc:2369] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:17.958617 32056 tablet.cc:2369] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:18.047381 32087 raft_consensus.cc:482] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:18.047528 32087 raft_consensus.cc:504] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.047996 32087 leader_election.cc:288] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515), fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897)
I20260811 11:10:18.051211 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.048330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:42995 (local address 127.29.145.1:34515)
0811 11:10:18.048405 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.048408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.048419 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:18.048455 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.048456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.048466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.048546 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.048801 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.049328 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.050395 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.050803 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.050861 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.050869 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.051088 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.051137 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:10:18.051211 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.048215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:34515 (local address 127.29.145.3:42995)
0811 11:10:18.048275 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.048289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.048339 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.048561 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.048562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.048564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.048683 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.048686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.049422 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.049423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.050306 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.050309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.050904 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.050906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.051011 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.051016 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.051070 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.051112 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":12}
I20260811 11:10:18.051357 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.048434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:54827 (local address 127.29.145.2:41897)
0811 11:10:18.048508 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.048510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.048641 (+ 131us) server_negotiation.cc:407] Connection header received
0811 11:10:18.048689 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.048690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.048702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.048756 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.048903 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.049408 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.050515 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.050912 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.050998 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.051007 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.051225 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.051276 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:18.051548 32088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.048354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:41897 (local address 127.29.145.3:54827)
0811 11:10:18.048564 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.048576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.048642 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.048764 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.048765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.048766 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.048842 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.048844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.049497 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.049498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.050403 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.050405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.051007 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.051008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.051105 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.051109 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.051166 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.051204 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":141,"threads_started":1}
I20260811 11:10:18.051573 31687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06a65d76c3b544969fb7671409254bd5" candidate_uuid: "e839036714e84ea2bb80d7a3e40e841f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" is_pre_election: true
I20260811 11:10:18.051743 31687 raft_consensus.cc:2454] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e839036714e84ea2bb80d7a3e40e841f in term 0.
I20260811 11:10:18.051920 31846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06a65d76c3b544969fb7671409254bd5" candidate_uuid: "e839036714e84ea2bb80d7a3e40e841f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa05721a3c9342f0869a6c824936d7fd" is_pre_election: true
I20260811 11:10:18.052075 31846 raft_consensus.cc:2454] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e839036714e84ea2bb80d7a3e40e841f in term 0.
I20260811 11:10:18.052066 31927 leader_election.cc:302] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, e839036714e84ea2bb80d7a3e40e841f; no voters:
I20260811 11:10:18.052205 32087 raft_consensus.cc:2781] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:18.052285 32087 raft_consensus.cc:482] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:18.052312 32087 raft_consensus.cc:3037] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.053436 32087 raft_consensus.cc:504] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.053583 32087 leader_election.cc:288] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [CANDIDATE]: Term 1 election: Requested vote from peers c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515), fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897)
I20260811 11:10:18.053797 31687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06a65d76c3b544969fb7671409254bd5" candidate_uuid: "e839036714e84ea2bb80d7a3e40e841f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c160d87cb8b040bb85b9fd6378dd40e4"
I20260811 11:10:18.053803 31846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06a65d76c3b544969fb7671409254bd5" candidate_uuid: "e839036714e84ea2bb80d7a3e40e841f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa05721a3c9342f0869a6c824936d7fd"
I20260811 11:10:18.053905 31846 raft_consensus.cc:3037] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.053906 31687 raft_consensus.cc:3037] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.054742 31846 raft_consensus.cc:2454] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e839036714e84ea2bb80d7a3e40e841f in term 1.
I20260811 11:10:18.054742 31687 raft_consensus.cc:2454] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e839036714e84ea2bb80d7a3e40e841f in term 1.
I20260811 11:10:18.054924 31927 leader_election.cc:302] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, e839036714e84ea2bb80d7a3e40e841f; no voters:
I20260811 11:10:18.055064 32087 raft_consensus.cc:2781] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:18.055226 32087 raft_consensus.cc:686] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 1 LEADER]: Becoming Leader. State: Replica: e839036714e84ea2bb80d7a3e40e841f, State: Running, Role: LEADER
I20260811 11:10:18.055336 32087 consensus_queue.cc:237] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.056239 31505 catalog_manager.cc:5495] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f reported cstate change: term changed from 0 to 1, leader changed from <none> to e839036714e84ea2bb80d7a3e40e841f (127.29.145.3). New cstate: current_term: 1 leader_uuid: "e839036714e84ea2bb80d7a3e40e841f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } health_report { overall_health: HEALTHY } } }
I20260811 11:10:18.110877 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.108038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.29.145.3:38109)
0811 11:10:18.108096 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.108097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.108114 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:10:18.108162 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.108162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.108180 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.108254 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:18.108464 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.108948 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.109963 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.110179 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.110257 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.110471 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.110586 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.110686 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.110738 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.110762 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:10:18.112833 31846 raft_consensus.cc:1261] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Refusing update from remote peer e839036714e84ea2bb80d7a3e40e841f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.112915 31687 raft_consensus.cc:1261] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Refusing update from remote peer e839036714e84ea2bb80d7a3e40e841f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.113196 32087 consensus_queue.cc:1035] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.113381 32089 consensus_queue.cc:1035] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.116889 32094 mvcc.cc:205] Tried to move back new op lower bound from 7317285347787173888 to 7317285347555287040. Current Snapshot: MvccSnapshot[applied={T|T < 7317285347787173888}]
I20260811 11:10:18.119496 31506 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:18.119614 31506 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:18.120700 31506 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Sending DeleteTablet for 3 replicas of tablet 06a65d76c3b544969fb7671409254bd5
I20260811 11:10:18.121172 31977 tablet_service.cc:1514] Processing DeleteTablet for tablet 06a65d76c3b544969fb7671409254bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:37850
I20260811 11:10:18.121193 31666 tablet_service.cc:1514] Processing DeleteTablet for tablet 06a65d76c3b544969fb7671409254bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:51594
I20260811 11:10:18.121219 31823 tablet_service.cc:1514] Processing DeleteTablet for tablet 06a65d76c3b544969fb7671409254bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:54444
I20260811 11:10:18.121533 32106 tablet_replica.cc:330] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: stopping tablet replica
I20260811 11:10:18.121563 32105 tablet_replica.cc:330] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: stopping tablet replica
I20260811 11:10:18.121687 32105 raft_consensus.cc:2229] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:18.121691 32106 raft_consensus.cc:2229] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:18.121841 32105 raft_consensus.cc:2258] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.121780 31506 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:18.121965 31506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:18.121996 32104 tablet_replica.cc:330] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: stopping tablet replica
I20260811 11:10:18.122084 32104 raft_consensus.cc:2229] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:18.122197 32104 raft_consensus.cc:2258] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.122224 32106 raft_consensus.cc:2258] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.122786 32106 ts_tablet_manager.cc:1905] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.122802 32104 ts_tablet_manager.cc:1905] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.123726 31666 tablet_service.cc:1467] Processing CreateTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 (DEFAULT_TABLE table=test-table [id=a9614be9a55748e79d3c9469a2a8afd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:18.123726 31977 tablet_service.cc:1467] Processing CreateTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 (DEFAULT_TABLE table=test-table [id=a9614be9a55748e79d3c9469a2a8afd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:18.123860 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40427d22b72d4bd19deb4ac325ca5ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.123864 31666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40427d22b72d4bd19deb4ac325ca5ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.124234 32105 ts_tablet_manager.cc:1905] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.125200 32083 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Bootstrap starting.
I20260811 11:10:18.125298 32080 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Bootstrap starting.
I20260811 11:10:18.125656 32083 tablet_bootstrap.cc:654] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.125734 32080 tablet_bootstrap.cc:654] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.126199 32105 ts_tablet_manager.cc:1918] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:18.126281 32105 log.cc:1196] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/06a65d76c3b544969fb7671409254bd5
I20260811 11:10:18.126494 31823 tablet_service.cc:1467] Processing CreateTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 (DEFAULT_TABLE table=test-table [id=a9614be9a55748e79d3c9469a2a8afd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:18.126581 32105 ts_tablet_manager.cc:1939] T 06a65d76c3b544969fb7671409254bd5 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting consensus metadata
I20260811 11:10:18.126613 31823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40427d22b72d4bd19deb4ac325ca5ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.127110 32106 ts_tablet_manager.cc:1918] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:18.127177 32106 log.cc:1196] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/06a65d76c3b544969fb7671409254bd5
I20260811 11:10:18.127260 32080 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: No bootstrap required, opened a new log
I20260811 11:10:18.127303 32080 ts_tablet_manager.cc:1397] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:18.127430 32106 ts_tablet_manager.cc:1939] T 06a65d76c3b544969fb7671409254bd5 P e839036714e84ea2bb80d7a3e40e841f: Deleting consensus metadata
I20260811 11:10:18.127683 32080 raft_consensus.cc:348] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.127760 32080 raft_consensus.cc:374] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.127779 32080 raft_consensus.cc:729] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c160d87cb8b040bb85b9fd6378dd40e4, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.127817 32080 consensus_queue.cc:260] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.127823 31487 catalog_manager.cc:4841] TS e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109): tablet 06a65d76c3b544969fb7671409254bd5 (table test-table [id=43452ade89254be5be328d0f2a11117b]) successfully deleted
I20260811 11:10:18.127933 32080 ts_tablet_manager.cc:1428] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:18.128022 32079 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Bootstrap starting.
I20260811 11:10:18.128392 32079 tablet_bootstrap.cc:654] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.128475 32083 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: No bootstrap required, opened a new log
I20260811 11:10:18.128518 32083 ts_tablet_manager.cc:1397] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 11:10:18.128647 32083 raft_consensus.cc:348] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.128719 32083 raft_consensus.cc:374] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.128741 32083 raft_consensus.cc:729] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e839036714e84ea2bb80d7a3e40e841f, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.128780 32083 consensus_queue.cc:260] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.128870 32083 ts_tablet_manager.cc:1428] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:18.129568 31488 catalog_manager.cc:4841] TS c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515): tablet 06a65d76c3b544969fb7671409254bd5 (table test-table [id=43452ade89254be5be328d0f2a11117b]) successfully deleted
I20260811 11:10:18.129917 32079 tablet_bootstrap.cc:492] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: No bootstrap required, opened a new log
I20260811 11:10:18.129995 32079 ts_tablet_manager.cc:1397] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:18.130151 32079 raft_consensus.cc:348] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.130225 32079 raft_consensus.cc:374] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.130260 32079 raft_consensus.cc:729] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa05721a3c9342f0869a6c824936d7fd, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.130311 32079 consensus_queue.cc:260] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.130414 32079 ts_tablet_manager.cc:1428] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:18.131692 32104 ts_tablet_manager.cc:1918] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:18.131790 32104 log.cc:1196] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/06a65d76c3b544969fb7671409254bd5
I20260811 11:10:18.132465 32104 ts_tablet_manager.cc:1939] T 06a65d76c3b544969fb7671409254bd5 P fa05721a3c9342f0869a6c824936d7fd: Deleting consensus metadata
I20260811 11:10:18.132884 31488 catalog_manager.cc:4841] TS fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897): tablet 06a65d76c3b544969fb7671409254bd5 (table test-table [id=43452ade89254be5be328d0f2a11117b]) successfully deleted
I20260811 11:10:18.343639 32086 raft_consensus.cc:482] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:18.343750 32086 raft_consensus.cc:504] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.344105 32086 leader_election.cc:288] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515), e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109)
I20260811 11:10:18.347204 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.344403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:51991 (local address 127.29.145.1:34515)
0811 11:10:18.344455 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.344457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.344532 (+ 75us) server_negotiation.cc:407] Connection header received
0811 11:10:18.344639 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.344640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.344649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.344714 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.344881 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.345369 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346464 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.346819 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346916 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.346928 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.347083 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.347123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:18.347234 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.344424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:53653 (local address 127.29.145.3:38109)
0811 11:10:18.344474 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.344476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.344636 (+ 160us) server_negotiation.cc:407] Connection header received
0811 11:10:18.344710 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.344712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.344732 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.344793 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.344952 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.345436 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346470 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.346840 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346926 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.346936 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.347119 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.347160 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:10:18.347257 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.344320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:34515 (local address 127.29.145.2:51991)
0811 11:10:18.344458 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.344470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.344542 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.344720 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.344721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.344723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.344803 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.344810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.345468 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.345469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.346371 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.346374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.346938 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.347007 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.347010 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.347076 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.347124 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260811 11:10:18.347474 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.344362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:38109 (local address 127.29.145.2:53653)
0811 11:10:18.344517 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.344529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.344641 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.344797 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.344798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.344800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.344877 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.344879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.345523 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.345524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.346369 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.346371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.346945 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.346947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.347021 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.347024 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.347106 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.347148 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260811 11:10:18.347533 31687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40427d22b72d4bd19deb4ac325ca5ca4" candidate_uuid: "fa05721a3c9342f0869a6c824936d7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" is_pre_election: true
I20260811 11:10:18.347645 31687 raft_consensus.cc:2454] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa05721a3c9342f0869a6c824936d7fd in term 0.
I20260811 11:10:18.347854 31766 leader_election.cc:302] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, fa05721a3c9342f0869a6c824936d7fd; no voters:
I20260811 11:10:18.347833 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40427d22b72d4bd19deb4ac325ca5ca4" candidate_uuid: "fa05721a3c9342f0869a6c824936d7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e839036714e84ea2bb80d7a3e40e841f" is_pre_election: true
I20260811 11:10:18.347972 32001 raft_consensus.cc:2454] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa05721a3c9342f0869a6c824936d7fd in term 0.
I20260811 11:10:18.348153 32086 raft_consensus.cc:2781] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:18.348217 32086 raft_consensus.cc:482] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:18.348250 32086 raft_consensus.cc:3037] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.348973 32086 raft_consensus.cc:504] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.349160 32086 leader_election.cc:288] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [CANDIDATE]: Term 1 election: Requested vote from peers c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515), e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109)
I20260811 11:10:18.349291 31687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40427d22b72d4bd19deb4ac325ca5ca4" candidate_uuid: "fa05721a3c9342f0869a6c824936d7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c160d87cb8b040bb85b9fd6378dd40e4"
I20260811 11:10:18.349334 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40427d22b72d4bd19deb4ac325ca5ca4" candidate_uuid: "fa05721a3c9342f0869a6c824936d7fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e839036714e84ea2bb80d7a3e40e841f"
I20260811 11:10:18.349395 31687 raft_consensus.cc:3037] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.349406 32001 raft_consensus.cc:3037] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.350142 31687 raft_consensus.cc:2454] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa05721a3c9342f0869a6c824936d7fd in term 1.
I20260811 11:10:18.350144 32001 raft_consensus.cc:2454] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa05721a3c9342f0869a6c824936d7fd in term 1.
I20260811 11:10:18.350301 31766 leader_election.cc:302] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, fa05721a3c9342f0869a6c824936d7fd; no voters:
I20260811 11:10:18.350399 32086 raft_consensus.cc:2781] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:18.350539 32086 raft_consensus.cc:686] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 1 LEADER]: Becoming Leader. State: Replica: fa05721a3c9342f0869a6c824936d7fd, State: Running, Role: LEADER
I20260811 11:10:18.350602 32086 consensus_queue.cc:237] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.351446 31506 catalog_manager.cc:5495] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd reported cstate change: term changed from 0 to 1, leader changed from <none> to fa05721a3c9342f0869a6c824936d7fd (127.29.145.2). New cstate: current_term: 1 leader_uuid: "fa05721a3c9342f0869a6c824936d7fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:10:18 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:18.398491 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.390718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37463 (local address 127.0.0.1:47842)
0811 11:10:18.390921 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.390937 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.391007 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.391212 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.391213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.391516 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:18.391566 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.391568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.392284 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.392285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.392818 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.392819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.393218 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.393219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.393269 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.393796 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:18.393801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:18.395367 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:18.397302 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:18.397310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:18.397312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:18.397510 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:18.397707 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:18.397708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:18.397708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:18.397784 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:18.398178 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:18.398185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:18.398231 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.398298 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.398406 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260811 11:10:18.398581 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.390778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.29.145.62:37463)
0811 11:10:18.391037 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.391039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.391049 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:18.391074 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.391074 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.391087 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.391194 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:18.391647 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.392166 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.392906 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.393099 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.393189 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.395474 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:18.395476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:18.395476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:18.395483 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:18.397217 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.397582 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.397583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.397583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.397642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.397906 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.397907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.397907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.398035 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:18.398194 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.398319 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.398473 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":227,"thread_start_us":124,"threads_started":1}
I20260811 11:10:18.401388 31506 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:47842:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:10:18.401685 31506 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:18.403333 31977 tablet_service.cc:1467] Processing CreateTablet for tablet 319123d92ab14e73a5395654c59680e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34eadb6d30b2458fa5e5c62307b5c031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:18.403434 31666 tablet_service.cc:1467] Processing CreateTablet for tablet 319123d92ab14e73a5395654c59680e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34eadb6d30b2458fa5e5c62307b5c031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:18.403522 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319123d92ab14e73a5395654c59680e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.403546 31823 tablet_service.cc:1467] Processing CreateTablet for tablet 319123d92ab14e73a5395654c59680e1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34eadb6d30b2458fa5e5c62307b5c031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:18.403604 31666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319123d92ab14e73a5395654c59680e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.403699 31823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 319123d92ab14e73a5395654c59680e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:18.406302 32083 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: Bootstrap starting.
I20260811 11:10:18.406332 32079 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: Bootstrap starting.
I20260811 11:10:18.406347 32080 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: Bootstrap starting.
I20260811 11:10:18.406724 32083 tablet_bootstrap.cc:654] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.406742 32080 tablet_bootstrap.cc:654] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.406832 32079 tablet_bootstrap.cc:654] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:18.407423 32080 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: No bootstrap required, opened a new log
I20260811 11:10:18.407475 32080 ts_tablet_manager.cc:1397] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:18.407702 32080 raft_consensus.cc:348] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.407831 32080 raft_consensus.cc:374] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.407866 32080 raft_consensus.cc:729] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c160d87cb8b040bb85b9fd6378dd40e4, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.407891 32079 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: No bootstrap required, opened a new log
I20260811 11:10:18.407919 32080 consensus_queue.cc:260] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.408007 32079 ts_tablet_manager.cc:1397] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:18.408056 32080 ts_tablet_manager.cc:1428] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:18.408154 32084 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.408174 32079 raft_consensus.cc:348] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.408257 32079 raft_consensus.cc:374] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.408260 32084 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.408282 32079 raft_consensus.cc:729] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa05721a3c9342f0869a6c824936d7fd, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.408330 32083 tablet_bootstrap.cc:492] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: No bootstrap required, opened a new log
I20260811 11:10:18.408329 32079 consensus_queue.cc:260] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.408382 32083 ts_tablet_manager.cc:1397] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:18.408416 32079 ts_tablet_manager.cc:1428] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:18.408474 32086 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.408515 32083 raft_consensus.cc:348] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.408561 32086 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.408587 32083 raft_consensus.cc:374] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:18.408612 32083 raft_consensus.cc:729] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e839036714e84ea2bb80d7a3e40e841f, State: Initialized, Role: FOLLOWER
I20260811 11:10:18.408686 32083 consensus_queue.cc:260] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.408807 32083 ts_tablet_manager.cc:1428] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:18.408897 32101 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.409003 32101 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.504510 32084 raft_consensus.cc:482] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:18.504633 32084 raft_consensus.cc:504] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.505295 32084 leader_election.cc:288] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897), e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109)
I20260811 11:10:18.508670 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.505593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:46267 (local address 127.29.145.3:38109)
0811 11:10:18.505646 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.505648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.505832 (+ 184us) server_negotiation.cc:407] Connection header received
0811 11:10:18.505952 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.505953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.505963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.506032 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.506279 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.506847 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.507908 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.508293 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.508382 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.508392 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.508554 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.508595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:10:18.508781 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.505546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:54143 (local address 127.29.145.2:41897)
0811 11:10:18.505617 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.505619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.505850 (+ 231us) server_negotiation.cc:407] Connection header received
0811 11:10:18.505978 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.505979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.505996 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.506072 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:18.506261 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.506950 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.508008 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.508396 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.508484 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.508493 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.508660 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.508697 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:10:18.508713 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.505522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:38109 (local address 127.29.145.1:46267)
0811 11:10:18.505753 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.505763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.505865 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.506037 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.506038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.506040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.506153 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.506155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.506931 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.506932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.507818 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.507820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.508380 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.508380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.508459 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.508463 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.508528 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.508572 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":22,"thread_start_us":147,"threads_started":1}
I20260811 11:10:18.509048 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.505500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:41897 (local address 127.29.145.1:54143)
0811 11:10:18.505742 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.505753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.505901 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.506078 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.506079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.506082 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:18.506170 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.506172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.507071 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.507072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.507903 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.507905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.508483 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.508484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.508571 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.508575 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.508652 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.508700 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":2,"thread_start_us":193,"threads_started":1}
I20260811 11:10:18.509194 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319123d92ab14e73a5395654c59680e1" candidate_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e839036714e84ea2bb80d7a3e40e841f" is_pre_election: true
I20260811 11:10:18.509311 32001 raft_consensus.cc:2454] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c160d87cb8b040bb85b9fd6378dd40e4 in term 0.
I20260811 11:10:18.509308 31846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319123d92ab14e73a5395654c59680e1" candidate_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa05721a3c9342f0869a6c824936d7fd" is_pre_election: true
I20260811 11:10:18.509389 31846 raft_consensus.cc:2454] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c160d87cb8b040bb85b9fd6378dd40e4 in term 0.
I20260811 11:10:18.509492 31608 leader_election.cc:302] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, e839036714e84ea2bb80d7a3e40e841f; no voters:
I20260811 11:10:18.509660 32084 raft_consensus.cc:2781] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:18.509704 32084 raft_consensus.cc:482] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:18.509720 32084 raft_consensus.cc:3037] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.510484 32084 raft_consensus.cc:504] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.510632 32084 leader_election.cc:288] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [CANDIDATE]: Term 1 election: Requested vote from peers fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897), e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109)
I20260811 11:10:18.510767 31846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319123d92ab14e73a5395654c59680e1" candidate_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa05721a3c9342f0869a6c824936d7fd"
I20260811 11:10:18.510794 32001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "319123d92ab14e73a5395654c59680e1" candidate_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e839036714e84ea2bb80d7a3e40e841f"
I20260811 11:10:18.510845 31846 raft_consensus.cc:3037] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.510857 32001 raft_consensus.cc:3037] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:18.511623 32001 raft_consensus.cc:2454] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c160d87cb8b040bb85b9fd6378dd40e4 in term 1.
I20260811 11:10:18.511623 31846 raft_consensus.cc:2454] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c160d87cb8b040bb85b9fd6378dd40e4 in term 1.
I20260811 11:10:18.511852 31609 leader_election.cc:302] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c160d87cb8b040bb85b9fd6378dd40e4, fa05721a3c9342f0869a6c824936d7fd; no voters:
I20260811 11:10:18.511999 32084 raft_consensus.cc:2781] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:18.512216 32084 raft_consensus.cc:686] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 LEADER]: Becoming Leader. State: Replica: c160d87cb8b040bb85b9fd6378dd40e4, State: Running, Role: LEADER
I20260811 11:10:18.512305 32084 consensus_queue.cc:237] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } }
I20260811 11:10:18.512694 32130 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: TxnStatusTablet state changed. Reason: New leader c160d87cb8b040bb85b9fd6378dd40e4. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.512817 32130 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:18.513155 31506 catalog_manager.cc:5495] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1). New cstate: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:18.513303 32133 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:10:18.513437 32133 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:10:18.524773 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.522194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:34515 (local address 127.0.0.1:51610)
0811 11:10:18.522271 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.522282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.522337 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.522531 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.522533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.522534 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:18.522585 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.522587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.523271 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.523273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.523843 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.523845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.524208 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.524209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.524268 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.524285 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.524576 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.524586 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.524649 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.524701 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:10:18.524773 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.522277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51610 (local address 127.29.145.1:34515)
0811 11:10:18.522325 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.522328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.522337 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:18.522422 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.522423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.522443 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.522527 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:18.522673 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.523178 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.523929 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.524132 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.524208 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.524400 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.524507 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.524590 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.524654 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.524677 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:18.580021 31846 raft_consensus.cc:1261] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Refusing update from remote peer c160d87cb8b040bb85b9fd6378dd40e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.580073 32001 raft_consensus.cc:1261] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Refusing update from remote peer c160d87cb8b040bb85b9fd6378dd40e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.580359 32130 consensus_queue.cc:1035] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.580515 32084 consensus_queue.cc:1035] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.581284 32101 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: TxnStatusTablet state changed. Reason: New leader c160d87cb8b040bb85b9fd6378dd40e4. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.581386 32101 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.581295 32086 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: TxnStatusTablet state changed. Reason: New leader c160d87cb8b040bb85b9fd6378dd40e4. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.581420 32086 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.582262 32084 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.582369 32084 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:18.582432 32084 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.582495 32084 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P c160d87cb8b040bb85b9fd6378dd40e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:18.582842 32092 mvcc.cc:205] Tried to move back new op lower bound from 7317285349701058560 to 7317285349427118080. Current Snapshot: MvccSnapshot[applied={T|T < 7317285349701058560}]
I20260811 11:10:18.582950 32086 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.583046 32086 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P fa05721a3c9342f0869a6c824936d7fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.582978 32095 mvcc.cc:205] Tried to move back new op lower bound from 7317285349701058560 to 7317285349427118080. Current Snapshot: MvccSnapshot[applied={T|T < 7317285349701058560}]
I20260811 11:10:18.583657 32101 tablet_replica.cc:439] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 } } peers { permanent_uuid: "fa05721a3c9342f0869a6c824936d7fd" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 41897 } } peers { permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 } } }
I20260811 11:10:18.583782 32101 tablet_replica.cc:442] T 319123d92ab14e73a5395654c59680e1 P e839036714e84ea2bb80d7a3e40e841f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:18.587826 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.585114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.29.145.2:41897)
0811 11:10:18.585180 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.585182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.585190 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:18.585229 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.585230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.585248 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.585330 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:18.585494 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.586167 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.586977 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.587182 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.587273 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.587446 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.587548 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.587641 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.587717 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.587739 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:10:18.593101 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.590648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:40371 (local address 127.29.145.1:34515)
0811 11:10:18.590710 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.590712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.590865 (+ 153us) server_negotiation.cc:407] Connection header received
0811 11:10:18.590938 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.590939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.590949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.591010 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:18.591151 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.591610 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.592396 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.592589 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.592666 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.592840 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.592878 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.592933 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.593001 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.593022 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:18.593438 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.590591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:34515 (local address 127.29.145.2:40371)
0811 11:10:18.590790 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.590800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.590883 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.591009 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.591009 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.591010 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:18.591077 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.591078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.591697 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.591698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.592303 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.592305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.592666 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.592667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.592733 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.592756 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.592948 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.592950 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.592988 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.593034 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260811 11:10:18.596284 32001 raft_consensus.cc:1261] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Refusing update from remote peer fa05721a3c9342f0869a6c824936d7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.596314 31687 raft_consensus.cc:1261] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Refusing update from remote peer fa05721a3c9342f0869a6c824936d7fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:18.596613 32086 consensus_queue.cc:1035] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e839036714e84ea2bb80d7a3e40e841f" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.596774 32114 consensus_queue.cc:1035] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c160d87cb8b040bb85b9fd6378dd40e4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:18.607170 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.603785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:41897 (local address 127.29.145.1:46277)
0811 11:10:18.603984 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:18.603996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:18.604068 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:18.604227 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:18.604228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:18.604230 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:18.604318 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.604319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.605271 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.605272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.606205 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:18.606209 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.606709 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.606710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:18.606795 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.606815 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.607014 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.607016 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:18.607052 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 11:10:18.607092 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260811 11:10:18.607172 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.604067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:46277 (local address 127.29.145.2:41897)
0811 11:10:18.604113 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.604115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.604123 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:18.604148 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.604149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.604159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.604220 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:18.604395 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.605181 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.606292 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.606619 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.606700 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.606913 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:18.606950 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:18.607007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.607056 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.607078 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:18.617138 31505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:18.617280 31505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:18.617993 31505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d439cefb8054acaa152011843eaeebd: Sending DeleteTablet for 3 replicas of tablet 40427d22b72d4bd19deb4ac325ca5ca4
I20260811 11:10:18.618361 31823 tablet_service.cc:1514] Processing DeleteTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:54444
I20260811 11:10:18.618389 31977 tablet_service.cc:1514] Processing DeleteTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:37850
I20260811 11:10:18.618495 32104 tablet_replica.cc:330] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: stopping tablet replica
I20260811 11:10:18.618546 32104 raft_consensus.cc:2229] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:18.618643 32104 raft_consensus.cc:2258] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.618722 32106 tablet_replica.cc:330] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: stopping tablet replica
I20260811 11:10:18.618783 32106 raft_consensus.cc:2229] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:18.618824 32106 raft_consensus.cc:2258] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.618901 31666 tablet_service.cc:1514] Processing DeleteTablet for tablet 40427d22b72d4bd19deb4ac325ca5ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:18 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:51594
I20260811 11:10:18.619005 32104 ts_tablet_manager.cc:1905] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.620452 32105 tablet_replica.cc:330] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: stopping tablet replica
I20260811 11:10:18.620535 32105 raft_consensus.cc:2229] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:18.620579 32105 raft_consensus.cc:2258] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:18.620599 32106 ts_tablet_manager.cc:1905] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.620831 32104 ts_tablet_manager.cc:1918] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:18.620887 32104 log.cc:1196] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/40427d22b72d4bd19deb4ac325ca5ca4
I20260811 11:10:18.621433 32104 ts_tablet_manager.cc:1939] T 40427d22b72d4bd19deb4ac325ca5ca4 P fa05721a3c9342f0869a6c824936d7fd: Deleting consensus metadata
I20260811 11:10:18.621879 31488 catalog_manager.cc:4841] TS fa05721a3c9342f0869a6c824936d7fd (127.29.145.2:41897): tablet 40427d22b72d4bd19deb4ac325ca5ca4 (table test-table [id=a9614be9a55748e79d3c9469a2a8afd3]) successfully deleted
I20260811 11:10:18.621989 32105 ts_tablet_manager.cc:1905] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:18.623301 32106 ts_tablet_manager.cc:1918] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:18.623373 32106 log.cc:1196] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/40427d22b72d4bd19deb4ac325ca5ca4
I20260811 11:10:18.623590 32106 ts_tablet_manager.cc:1939] T 40427d22b72d4bd19deb4ac325ca5ca4 P e839036714e84ea2bb80d7a3e40e841f: Deleting consensus metadata
I20260811 11:10:18.624310 32105 ts_tablet_manager.cc:1918] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:18.624375 32105 log.cc:1196] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/40427d22b72d4bd19deb4ac325ca5ca4
I20260811 11:10:18.624353 31487 catalog_manager.cc:4841] TS e839036714e84ea2bb80d7a3e40e841f (127.29.145.3:38109): tablet 40427d22b72d4bd19deb4ac325ca5ca4 (table test-table [id=a9614be9a55748e79d3c9469a2a8afd3]) successfully deleted
I20260811 11:10:18.624596 32105 ts_tablet_manager.cc:1939] T 40427d22b72d4bd19deb4ac325ca5ca4 P c160d87cb8b040bb85b9fd6378dd40e4: Deleting consensus metadata
I20260811 11:10:18.624950 31488 catalog_manager.cc:4841] TS c160d87cb8b040bb85b9fd6378dd40e4 (127.29.145.1:34515): tablet 40427d22b72d4bd19deb4ac325ca5ca4 (table test-table [id=a9614be9a55748e79d3c9469a2a8afd3]) successfully deleted
Aug 11 11:10:18 dist-test-slave-69rg krb5kdc[31454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:18.636595 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.619208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.29.145.1:34515)
0811 11:10:18.619258 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.619260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.626053 (+ 6793us) server_negotiation.cc:407] Connection header received
0811 11:10:18.626112 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.626113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.626126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.626193 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:18.627004 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.627754 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.628850 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.629175 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.629251 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.632960 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:18.632968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:18.632968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:18.632976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:18.635141 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.635587 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.635591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.635591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.635681 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.635923 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.635924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.635925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.636129 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:18.636314 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.636365 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.636474 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 11:10:18.636927 31731 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51620
I20260811 11:10:18.648478 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:18.638183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47846 (local address 127.29.145.62:37463)
0811 11:10:18.638258 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:18.638261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:18.640736 (+ 2475us) server_negotiation.cc:407] Connection header received
0811 11:10:18.640778 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:18.640779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:18.640792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:18.640864 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:18.641529 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.642425 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.643477 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:18.643798 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:18.643875 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:18.645410 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:18.645412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:18.645412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:18.645420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:18.647154 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.647556 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.647557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.647558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.647623 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:18.647863 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:18.647864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:18.647864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:18.648006 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:18.648116 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:18.648232 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:10:18.648378 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:10:18.648816 31505 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47846
I20260811 11:10:18.649593 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31591
I20260811 11:10:18.666676 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31749
I20260811 11:10:18.676347 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31909
I20260811 11:10:18.685706 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 31469
2026-08-11T11:10:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3097 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:18 dist-test-slave-69rg krb5kdc[32161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:18 dist-test-slave-69rg krb5kdc[32161](info): set up 2 sockets
Aug 11 11:10:18 dist-test-slave-69rg krb5kdc[32161](info): commencing operation
krb5kdc: starting...
W20260811 11:10:19.840924 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:19 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:19.980973 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34527
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:34527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:20.067679 32175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:20.067945 32175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:20.068027 32175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:20.069690 32175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:20.069759 32175 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:10:20.069785 32175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:20.069832 32175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:20.069850 32175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:20.069869 32175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:20.072278 32175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:34527
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:20.072559 32175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:20.072845 32175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:20.078541 32184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:20.078558 32182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.078671 32175 server_base.cc:1034] running on GCE node
W20260811 11:10:20.078886 32181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.079166 32175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:20.079491 32175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:20.080670 32175 hybrid_clock.cc:648] HybridClock initialized: now 1786446620080642 us; error 59 us; skew 500 ppm
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:20.083438 32175 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:20.083896 32175 webserver.cc:460] Webserver started at http://127.29.145.62:44403/ using document root <none> and password file <none>
I20260811 11:10:20.084121 32175 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:20.084165 32175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:20.084257 32175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:20.085301 32175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "744764cb01d64c68b361ca4d423d87ac"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "45d69d867bd0747709ac7cee43ea1163"
server_key_iv: "a8d10f3f5c4c1dd447c877787cde40d3"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.085608 32175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "744764cb01d64c68b361ca4d423d87ac"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "45d69d867bd0747709ac7cee43ea1163"
server_key_iv: "a8d10f3f5c4c1dd447c877787cde40d3"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.087069 32175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:20.087850 32191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.088052 32175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.088130 32175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "744764cb01d64c68b361ca4d423d87ac"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "45d69d867bd0747709ac7cee43ea1163"
server_key_iv: "a8d10f3f5c4c1dd447c877787cde40d3"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.088195 32175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:20.110966 32175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:20.113466 32175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:20.113584 32175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:20.119091 32175 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:34527
I20260811 11:10:20.119158 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:34527 every 8 connection(s)
I20260811 11:10:20.119504 32175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:20.120091 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 32175
I20260811 11:10:20.120177 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:20.120268 32267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.120391 30276 external_mini_cluster.cc:1428] Setting key 6ffcb7ac51fa5e5d238656c469c03b49
I20260811 11:10:20.124128 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Bootstrap starting.
I20260811 11:10:20.124903 32267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.125226 32267 log.cc:824] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:20.126055 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: No bootstrap required, opened a new log
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.128057 32267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } }
I20260811 11:10:20.128257 32267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.128310 32267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744764cb01d64c68b361ca4d423d87ac, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.128445 32267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } }
I20260811 11:10:20.128530 32267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:20.128583 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:20.128643 32267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.129205 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } }
I20260811 11:10:20.129331 32267 leader_election.cc:302] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 744764cb01d64c68b361ca4d423d87ac; no voters:
I20260811 11:10:20.129544 32267 leader_election.cc:288] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:20.129581 32272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:20.129750 32272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [term 1 LEADER]: Becoming Leader. State: Replica: 744764cb01d64c68b361ca4d423d87ac, State: Running, Role: LEADER
I20260811 11:10:20.129874 32272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } }
I20260811 11:10:20.129884 32267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:20.131313 32274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744764cb01d64c68b361ca4d423d87ac. Latest consensus state: current_term: 1 leader_uuid: "744764cb01d64c68b361ca4d423d87ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } } }
I20260811 11:10:20.131428 32274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:20.131640 32273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "744764cb01d64c68b361ca4d423d87ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744764cb01d64c68b361ca4d423d87ac" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34527 } } }
I20260811 11:10:20.131742 32273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac [sys.catalog]: This master's current role is: LEADER
W20260811 11:10:20.132215 32288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:10:20.132277 32288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:10:20.132329 32289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:20.132483 32289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:20.132732 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.121029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.29.145.62:34527)
0811 11:10:20.121234 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.121243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.121282 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:10:20.121768 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.121775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.121873 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.122003 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.123015 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.124400 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.125530 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.126050 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.126172 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.128561 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.128565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.128565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.128578 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.130897 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.131309 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.131314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.131314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.131465 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.131728 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.131729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.131730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.132045 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.132224 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.132385 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.132532 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260811 11:10:20.134092 32289 catalog_manager.cc:1347] Generated new cluster ID: f4ed5e204faa441db61b0c1594c5d05d
I20260811 11:10:20.134164 32289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:20.150764 32289 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:20.151355 32289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:20.158434 32289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Generated new TSK 0
I20260811 11:10:20.158643 32289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:20.189899 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34527
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:20.281457 32294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:20.281649 32294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:20.281675 32294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:20.282864 32294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:20.282907 32294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:20.282929 32294 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:10:20.282977 32294 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:20.283008 32294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:20.284487 32294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:20.284720 32294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:20.284965 32294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:20.290019 32303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.290158 32294 server_base.cc:1034] running on GCE node
W20260811 11:10:20.290045 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:20.290027 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.290474 32294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:20.290726 32294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:20.291900 32294 hybrid_clock.cc:648] HybridClock initialized: now 1786446620291862 us; error 57 us; skew 500 ppm
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:20.294500 32294 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:20.294997 32294 webserver.cc:460] Webserver started at http://127.29.145.1:45269/ using document root <none> and password file <none>
I20260811 11:10:20.295224 32294 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:20.295266 32294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:20.295356 32294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:20.296414 32294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "4fbfb0afee1c4a5a03bb6ce1ac27bc9c"
server_key_iv: "10ac3cd6859a92765dd5aa27c4db1761"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.296721 32294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "4fbfb0afee1c4a5a03bb6ce1ac27bc9c"
server_key_iv: "10ac3cd6859a92765dd5aa27c4db1761"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.298105 32294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:10:20.298918 32312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.299197 32294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.299283 32294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "4fbfb0afee1c4a5a03bb6ce1ac27bc9c"
server_key_iv: "10ac3cd6859a92765dd5aa27c4db1761"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.299338 32294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:20.317584 32294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:20.320043 32294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:20.320163 32294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:20.320675 32294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:20.320712 32294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.320742 32294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:20.320802 32294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.328486 32294 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:39539
I20260811 11:10:20.328547 32442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:39539 every 8 connection(s)
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.328926 32294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:20.332346 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.321268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:34841 (local address 127.29.145.62:34527)
0811 11:10:20.321353 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.321355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.321500 (+ 145us) server_negotiation.cc:407] Connection header received
0811 11:10:20.322125 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.322127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.322142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.322221 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.323259 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.324205 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.325968 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.326332 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.326419 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.329288 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.329289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.329290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.329297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.330920 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.331390 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.331391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.331392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.331464 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.331782 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.331783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.331783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.331902 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.331959 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.332206 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.332282 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:10:20.332497 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.321167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.1:34841)
0811 11:10:20.321362 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.321404 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.322015 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.322359 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.322364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.322727 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.323061 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.323068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.324329 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.324330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.325862 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.325866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.326441 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.326442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.326576 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.327182 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.327192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.329169 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.330995 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.331001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.331003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.331303 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.331545 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.331546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.331546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.331705 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.331966 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.331971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.332088 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.332179 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.332307 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260811 11:10:20.333070 32443 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34527
I20260811 11:10:20.333170 32443 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:20.333370 32443 heartbeater.cc:511] Master 127.29.145.62:34527 requested a full tablet report, sending...
I20260811 11:10:20.333982 32212 ts_manager.cc:194] Registered new tserver with Master: 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539)
I20260811 11:10:20.334719 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:34841
I20260811 11:10:20.336261 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 32294
I20260811 11:10:20.336372 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:20.336544 30276 external_mini_cluster.cc:1428] Setting key 65959a85c4366070299146cb860d96b6
I20260811 11:10:20.340631 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.334164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.1:40629)
0811 11:10:20.334319 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.334330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.334392 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.335079 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.335081 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.335255 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.335307 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.335308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.335948 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.335949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.336535 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.336537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.336921 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.336922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.336984 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.337433 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.337439 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.337855 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.339562 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.339563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.339563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.339731 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.339956 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.339957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.339958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.340018 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.340282 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.340283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.340329 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.340409 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.340538 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260811 11:10:20.340626 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.334916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:40629 (local address 127.29.145.62:34527)
0811 11:10:20.334951 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.334953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.334961 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:20.334980 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.334980 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.334989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.335053 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.335385 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.335852 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.336635 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.336834 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.336919 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.337935 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.337937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.337937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.337943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.339473 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.339824 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.339825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.339825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.339881 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.340092 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.340093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.340094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.340206 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.340277 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.340448 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.340532 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:20.370664 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34527
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:20.455456 32454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:20.455677 32454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:20.455726 32454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:20.456884 32454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:20.456924 32454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:20.456945 32454 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:10:20.456957 32454 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:20.456974 32454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:20.458451 32454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:20.458666 32454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:20.458856 32454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:20.465276 32461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:20.465260 32460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.465562 32454 server_base.cc:1034] running on GCE node
W20260811 11:10:20.465269 32463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.465957 32454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:20.466250 32454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:20.467440 32454 hybrid_clock.cc:648] HybridClock initialized: now 1786446620467425 us; error 31 us; skew 500 ppm
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:20.470156 32454 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:20.470639 32454 webserver.cc:460] Webserver started at http://127.29.145.2:41597/ using document root <none> and password file <none>
I20260811 11:10:20.470903 32454 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:20.470949 32454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:20.471109 32454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:20.472267 32454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "be27f76659384b5bbb850ea85fb1fb98"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "1d0101bb591e240288e9a036b12ff254"
server_key_iv: "210d522e3d19a4bea130ad291911910f"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.472643 32454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "be27f76659384b5bbb850ea85fb1fb98"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "1d0101bb591e240288e9a036b12ff254"
server_key_iv: "210d522e3d19a4bea130ad291911910f"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.474098 32454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:20.474923 32470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.475111 32454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.475203 32454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "be27f76659384b5bbb850ea85fb1fb98"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "1d0101bb591e240288e9a036b12ff254"
server_key_iv: "210d522e3d19a4bea130ad291911910f"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.475263 32454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:20.492437 32454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:20.494807 32454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:20.494925 32454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:20.495450 32454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:20.495497 32454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.495522 32454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:20.495534 32454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.503211 32454 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:34719
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.503300 32605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:34719 every 8 connection(s)
I20260811 11:10:20.503602 32454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:20.506506 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.495983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41175 (local address 127.29.145.62:34527)
0811 11:10:20.496058 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.496062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.496248 (+ 186us) server_negotiation.cc:407] Connection header received
0811 11:10:20.496772 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.496773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.496786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.496861 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.497801 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.498733 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.500450 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.500798 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.500878 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.503830 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.503832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.503832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.503841 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.505033 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.505442 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.505444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.505444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.505501 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.505778 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.505779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.505779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.505931 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.506012 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.506319 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.506415 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:10:20.506664 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.495953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.2:41175)
0811 11:10:20.496111 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.496154 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.496660 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.496984 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.496990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.497388 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.497695 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.497703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.498859 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.498860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.500338 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.500342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.500902 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.500903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.501058 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.501770 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.501782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.503695 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.505124 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.505131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.505134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.505360 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.505589 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.505590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.505590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.505701 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.506002 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.506009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.506154 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.506272 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.506433 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260811 11:10:20.506983 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 32454
I20260811 11:10:20.507112 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:20.507193 32606 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34527
I20260811 11:10:20.507284 32606 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:20.507369 30276 external_mini_cluster.cc:1428] Setting key 372b2b9173340e28a2c38a1c9b05d87e
I20260811 11:10:20.507484 32606 heartbeater.cc:511] Master 127.29.145.62:34527 requested a full tablet report, sending...
I20260811 11:10:20.507943 32212 ts_manager.cc:194] Registered new tserver with Master: be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719)
I20260811 11:10:20.508472 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:41175
I20260811 11:10:20.514364 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.508351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:51565 (local address 127.29.145.62:34527)
0811 11:10:20.508392 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.508394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.508405 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:20.508468 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.508469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.508479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.508522 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.508914 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.509626 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.510393 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.510598 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.510675 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.511614 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.511615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.511615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.511622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.513086 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.513437 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.513439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.513439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.513500 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.513711 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.513712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.513712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.513940 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.514016 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.514181 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.514267 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:20.514377 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.508228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.2:51565)
0811 11:10:20.508354 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.508365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.508419 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.508547 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.508548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.508758 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.508824 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.508825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.509739 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.509740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.510297 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.510299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.510686 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.510688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.510763 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.511214 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.511219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.511500 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.513175 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.513176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.513176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.513361 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.513566 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.513567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.513567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.513635 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.514019 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.514021 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.514068 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.514154 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.514269 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:20.537696 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34527
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:20.622516 32615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:20.622763 32615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:20.622804 32615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:20.623952 32615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:20.624028 32615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:20.624064 32615 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:10:20.624092 32615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:20.624125 32615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:20.625597 32615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:20.625908 32615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:20.626156 32615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:20.631552 32624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.631615 32615 server_base.cc:1034] running on GCE node
W20260811 11:10:20.631552 32623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:20.631814 32626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:20.632054 32615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:20.632311 32615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:20.633484 32615 hybrid_clock.cc:648] HybridClock initialized: now 1786446620633476 us; error 35 us; skew 500 ppm
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:20.636044 32615 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:20.636485 32615 webserver.cc:460] Webserver started at http://127.29.145.3:36689/ using document root <none> and password file <none>
I20260811 11:10:20.636719 32615 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:20.636788 32615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:20.636952 32615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:20.638127 32615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "c638d309513c4457a17e2f3ecab38d5e"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "f36f6a376ac06fcd73d920be9ee8ce6f"
server_key_iv: "85e7bf2ee040800adb58c61d7acb60d4"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.638506 32615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "c638d309513c4457a17e2f3ecab38d5e"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "f36f6a376ac06fcd73d920be9ee8ce6f"
server_key_iv: "85e7bf2ee040800adb58c61d7acb60d4"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.639906 32615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.640718 32634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.640928 32615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.641039 32615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "c638d309513c4457a17e2f3ecab38d5e"
format_stamp: "Formatted at 2026-08-11 11:10:20 on dist-test-slave-69rg"
server_key: "f36f6a376ac06fcd73d920be9ee8ce6f"
server_key_iv: "85e7bf2ee040800adb58c61d7acb60d4"
server_key_version: "encryptionkey@0"
I20260811 11:10:20.641117 32615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:20.652530 32615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:20.654629 32615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:20.654811 32615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:20.655364 32615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:20.655433 32615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.655478 32615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:20.655522 32615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.662953 32615 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:44407
I20260811 11:10:20.663031 32761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:44407 every 8 connection(s)
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.663339 32615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:20.666872 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.655951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59459 (local address 127.29.145.62:34527)
0811 11:10:20.655999 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.656001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.656255 (+ 254us) server_negotiation.cc:407] Connection header received
0811 11:10:20.656786 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.656788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.656814 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.656898 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.657855 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.658725 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.660338 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.660684 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.660787 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.663896 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.663898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.663898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.663906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.665419 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.665771 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.665772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.665772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.665867 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.666157 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.666159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.666159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.666357 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.666435 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.666733 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.666805 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:20.667100 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.655932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.3:59459)
0811 11:10:20.656114 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.656168 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.656676 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.656997 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.657006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.657390 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.657723 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.657729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.658828 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.658829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.660252 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.660256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.660805 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.660806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.660964 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.661669 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.661681 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.663803 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.665490 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.665494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.665496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.665703 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.665950 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.665951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.665951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.666067 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.666467 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.666473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.666595 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.666705 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.666840 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":117,"threads_started":1}
I20260811 11:10:20.667871 32762 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34527
I20260811 11:10:20.667966 32762 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:20.668154 32762 heartbeater.cc:511] Master 127.29.145.62:34527 requested a full tablet report, sending...
I20260811 11:10:20.668567 32211 ts_manager.cc:194] Registered new tserver with Master: c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:20.669034 32211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:59459
I20260811 11:10:20.673429 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 32615
I20260811 11:10:20.673499 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:20.673684 30276 external_mini_cluster.cc:1428] Setting key d945401d40ea45e759f30a94b4c2e445
I20260811 11:10:20.674571 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:10:20.674996 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.669179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:57115 (local address 127.29.145.62:34527)
0811 11:10:20.669230 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.669232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.669492 (+ 260us) server_negotiation.cc:407] Connection header received
0811 11:10:20.669521 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.669522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.669531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.669581 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.669954 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.670416 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.671091 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.671280 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.671341 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.672195 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.672197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.672197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.672203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.673681 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.674108 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.674109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.674110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.674177 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.674451 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.674452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.674452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.674596 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.674674 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.674839 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.674917 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:10:20.675009 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.669132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.29.145.3:57115)
0811 11:10:20.669393 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.669405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.669479 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.669596 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.669596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.669802 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.669883 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.669886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.670508 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.670509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.671021 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.671023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.671357 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.671358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.671416 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.671809 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.671814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.672085 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.673786 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.673787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.673788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.674034 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.674259 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.674260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.674260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.674330 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.674676 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.674677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.674733 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.674790 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.674893 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":120,"threads_started":1}
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.700582 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.692934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.29.145.62:34527)
0811 11:10:20.693043 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.693044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.693054 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:20.693119 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.693120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.693130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.693236 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.693935 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.694445 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.695238 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.695438 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.695519 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.697969 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.697971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.697971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.697980 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.699349 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.699764 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.699766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.699766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.699828 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.700032 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.700033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.700034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.700159 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.700269 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.700390 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.700473 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 11:10:20.702445 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:20.704160 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:20.711272 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.706711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34719 (local address 127.0.0.1:55676)
0811 11:10:20.706840 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.706871 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.706949 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.707489 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.707493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.707496 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.707574 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.707576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.708597 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.708597 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.709596 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.709599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.710904 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.710905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.710989 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.711003 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.711100 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.711162 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260811 11:10:20.711313 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.706852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.29.145.2:34719)
0811 11:10:20.707054 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.707060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.707072 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:20.707116 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.707118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.707136 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.707220 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.707671 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.708515 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.709714 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.710805 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.710893 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.710960 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.711169 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.711211 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
I20260811 11:10:20.711478 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.707094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.29.145.3:44407)
0811 11:10:20.707341 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.707345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.707359 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:10:20.707718 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.707721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.707739 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.707833 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.707959 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.709111 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.710272 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.710820 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.710908 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.710967 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.711320 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.711365 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260811 11:10:20.711278 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.706949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:44407 (local address 127.0.0.1:55850)
0811 11:10:20.707299 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.707311 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.707359 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.707831 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.707832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.707834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.707878 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.707880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.709214 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.709215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.710180 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.710183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.711008 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.711009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.711076 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.711081 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.711131 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.711173 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":72,"threads_started":1}
I20260811 11:10:20.712121 315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.707141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39539 (local address 127.0.0.1:56280)
0811 11:10:20.707301 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.707311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.707359 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.707900 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.707901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.707903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.707943 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.707944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.709258 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.709259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.710659 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.710663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.711909 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.711910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.711969 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.711974 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.712022 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.712060 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260811 11:10:20.712380 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.707306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.29.145.1:39539)
0811 11:10:20.707465 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.707469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.707478 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:20.707513 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.707515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.707530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.707618 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.708011 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.709188 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.710780 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.711735 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.711816 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.711871 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.712232 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.712272 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260811 11:10:20.712357 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3d73340aa145d9b60a75adebe23843 (DEFAULT_TABLE table=test-table [id=f704868a005c4bda9ea657b4177b0dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:20.712486 32528 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3d73340aa145d9b60a75adebe23843 (DEFAULT_TABLE table=test-table [id=f704868a005c4bda9ea657b4177b0dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:20.712800 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3d73340aa145d9b60a75adebe23843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.712898 32528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3d73340aa145d9b60a75adebe23843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.713079 32370 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3d73340aa145d9b60a75adebe23843 (DEFAULT_TABLE table=test-table [id=f704868a005c4bda9ea657b4177b0dfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:20.713409 32370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3d73340aa145d9b60a75adebe23843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.716526 321 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Bootstrap starting.
I20260811 11:10:20.717264 321 tablet_bootstrap.cc:654] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.717252 322 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Bootstrap starting.
I20260811 11:10:20.717754 321 log.cc:824] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:20.718340 322 tablet_bootstrap.cc:654] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.718334 323 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Bootstrap starting.
I20260811 11:10:20.718695 322 log.cc:824] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:20.718801 321 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: No bootstrap required, opened a new log
I20260811 11:10:20.718876 321 ts_tablet_manager.cc:1397] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:20.718930 323 tablet_bootstrap.cc:654] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.719285 323 log.cc:824] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:20.719560 322 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: No bootstrap required, opened a new log
I20260811 11:10:20.719648 322 ts_tablet_manager.cc:1397] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:20.720036 323 tablet_bootstrap.cc:492] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: No bootstrap required, opened a new log
I20260811 11:10:20.720124 323 ts_tablet_manager.cc:1397] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:20.720949 321 raft_consensus.cc:348] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.721148 321 raft_consensus.cc:374] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.721186 321 raft_consensus.cc:729] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.721303 321 consensus_queue.cc:260] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.721597 321 ts_tablet_manager.cc:1428] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:10:20.721625 32606 heartbeater.cc:503] Master 127.29.145.62:34527 was elected leader, sending a full tablet report...
I20260811 11:10:20.721650 322 raft_consensus.cc:348] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.721799 322 raft_consensus.cc:374] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.721887 322 raft_consensus.cc:729] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.721762 323 raft_consensus.cc:348] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.721938 323 raft_consensus.cc:374] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.721990 323 raft_consensus.cc:729] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420a8ffea6d4380a8c96b0b94fd2da0, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.722056 322 consensus_queue.cc:260] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.722112 323 consensus_queue.cc:260] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.722357 322 ts_tablet_manager.cc:1428] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:20.722383 32762 heartbeater.cc:503] Master 127.29.145.62:34527 was elected leader, sending a full tablet report...
I20260811 11:10:20.722393 323 ts_tablet_manager.cc:1428] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:10:20.722517 32443 heartbeater.cc:503] Master 127.29.145.62:34527 was elected leader, sending a full tablet report...
W20260811 11:10:20.754204 32607 tablet.cc:2369] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:20.829730 32444 tablet.cc:2369] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:20.872002 328 raft_consensus.cc:482] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:20.872195 328 raft_consensus.cc:504] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.872632 328 leader_election.cc:288] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719), 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539)
I20260811 11:10:20.875730 330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.873004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39539 (local address 127.29.145.3:60711)
0811 11:10:20.873175 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.873185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.873234 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.873409 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.873410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.873412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.873478 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.873480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.874143 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.874144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.874886 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.874888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.875457 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.875458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.875536 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.875541 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.875600 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.875630 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260811 11:10:20.875747 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.873089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:60711 (local address 127.29.145.1:39539)
0811 11:10:20.873137 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.873139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.873288 (+ 149us) server_negotiation.cc:407] Connection header received
0811 11:10:20.873319 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.873320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.873331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.873386 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.873548 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.874031 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.875006 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.875380 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.875436 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.875443 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.875615 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.875660 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:10:20.875869 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.872963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:51687 (local address 127.29.145.2:34719)
0811 11:10:20.873017 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.873020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.873032 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:20.873107 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.873107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.873117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.873173 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.873396 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.874001 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.874958 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.875465 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.875537 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.875545 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.875743 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.875786 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 11:10:20.875852 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.872869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34719 (local address 127.29.145.3:51687)
0811 11:10:20.872955 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.872971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.873044 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.873184 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.873188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.873195 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.873295 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.873298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.874093 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.874094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.874866 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.874868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.875556 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.875557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.875658 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.875662 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.875722 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.875779 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 11:10:20.876343 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3d73340aa145d9b60a75adebe23843" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" is_pre_election: true
I20260811 11:10:20.876349 32552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3d73340aa145d9b60a75adebe23843" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be27f76659384b5bbb850ea85fb1fb98" is_pre_election: true
I20260811 11:10:20.876508 32552 raft_consensus.cc:2454] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 0.
I20260811 11:10:20.876508 32390 raft_consensus.cc:2454] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 0.
I20260811 11:10:20.876793 32635 leader_election.cc:302] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:20.876928 328 raft_consensus.cc:2781] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:20.877019 328 raft_consensus.cc:482] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:20.877054 328 raft_consensus.cc:3037] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.877792 328 raft_consensus.cc:504] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.877985 328 leader_election.cc:288] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 election: Requested vote from peers be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719), 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539)
I20260811 11:10:20.878172 32552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3d73340aa145d9b60a75adebe23843" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be27f76659384b5bbb850ea85fb1fb98"
I20260811 11:10:20.878252 32552 raft_consensus.cc:3037] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.878224 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3d73340aa145d9b60a75adebe23843" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
I20260811 11:10:20.878330 32390 raft_consensus.cc:3037] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.879004 32552 raft_consensus.cc:2454] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 1.
I20260811 11:10:20.879168 32390 raft_consensus.cc:2454] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 1.
I20260811 11:10:20.879235 32635 leader_election.cc:302] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be27f76659384b5bbb850ea85fb1fb98, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:20.879338 328 raft_consensus.cc:2781] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:20.879524 328 raft_consensus.cc:686] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 1 LEADER]: Becoming Leader. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Running, Role: LEADER
I20260811 11:10:20.879637 328 consensus_queue.cc:237] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } }
I20260811 11:10:20.880584 32212 catalog_manager.cc:5495] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to c638d309513c4457a17e2f3ecab38d5e (127.29.145.3). New cstate: current_term: 1 leader_uuid: "c638d309513c4457a17e2f3ecab38d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:10:20 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446620, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:20.913476 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.906604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34527 (local address 127.0.0.1:50638)
0811 11:10:20.906847 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.906858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.906925 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.907075 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.907076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.907393 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:20.907446 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.907448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.908189 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.908190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.908722 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.908723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.909047 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.909048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.909091 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.909649 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:20.909654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:20.911102 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:20.912531 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.912538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.912540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.912731 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.912921 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:20.912922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:20.912922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:20.912978 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:20.913185 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:20.913191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:20.913230 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.913288 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.913380 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":163,"threads_started":1}
I20260811 11:10:20.913547 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.906718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50638 (local address 127.29.145.62:34527)
0811 11:10:20.906788 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.906790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.906924 (+ 134us) server_negotiation.cc:407] Connection header received
0811 11:10:20.906988 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.906989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.907000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.907059 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:20.907520 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.908119 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.908801 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.908977 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.909032 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.911186 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:20.911188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:20.911188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:20.911194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:20.912456 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.912803 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.912804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.912804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.912856 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:20.913034 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:20.913035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:20.913035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:20.913132 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:20.913207 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.913340 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.913428 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":39}
W20260811 11:10:20.914179 32764 tablet.cc:2369] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:20.915786 32212 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:50638:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:10:20.916002 32212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:20.917703 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbefd36ff98448095f3dee8eeddcd89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=677536b55b5742dca1c5d228e30cbfef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:20.917665 32370 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbefd36ff98448095f3dee8eeddcd89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=677536b55b5742dca1c5d228e30cbfef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:20.917716 32528 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbefd36ff98448095f3dee8eeddcd89 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=677536b55b5742dca1c5d228e30cbfef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:20.917897 32528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbefd36ff98448095f3dee8eeddcd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.917898 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbefd36ff98448095f3dee8eeddcd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.917898 32370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbefd36ff98448095f3dee8eeddcd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:20.919046 322 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: Bootstrap starting.
I20260811 11:10:20.919422 323 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: Bootstrap starting.
I20260811 11:10:20.919497 322 tablet_bootstrap.cc:654] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.919833 323 tablet_bootstrap.cc:654] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.920156 321 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: Bootstrap starting.
I20260811 11:10:20.920306 322 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: No bootstrap required, opened a new log
I20260811 11:10:20.920367 322 ts_tablet_manager.cc:1397] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.920501 323 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: No bootstrap required, opened a new log
I20260811 11:10:20.920554 323 ts_tablet_manager.cc:1397] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:20.920533 322 raft_consensus.cc:348] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.920632 322 raft_consensus.cc:374] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.920661 322 raft_consensus.cc:729] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.920707 322 consensus_queue.cc:260] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.920706 323 raft_consensus.cc:348] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.920784 323 raft_consensus.cc:374] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.920812 323 raft_consensus.cc:729] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420a8ffea6d4380a8c96b0b94fd2da0, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.920804 322 ts_tablet_manager.cc:1428] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:20.920863 323 consensus_queue.cc:260] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.920909 321 tablet_bootstrap.cc:654] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:20.920959 323 ts_tablet_manager.cc:1428] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.920933 328 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.921003 328 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.921041 329 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.921144 329 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.921633 321 tablet_bootstrap.cc:492] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: No bootstrap required, opened a new log
I20260811 11:10:20.921701 321 ts_tablet_manager.cc:1397] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:20.921872 321 raft_consensus.cc:348] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.921941 321 raft_consensus.cc:374] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:20.921955 321 raft_consensus.cc:729] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Initialized, Role: FOLLOWER
I20260811 11:10:20.922019 321 consensus_queue.cc:260] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.922111 321 ts_tablet_manager.cc:1428] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:20.922184 327 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.922255 327 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.946210 327 raft_consensus.cc:482] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:20.946314 327 raft_consensus.cc:504] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.946718 327 leader_election.cc:288] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:20.950627 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.947009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:45633 (local address 127.29.145.1:39539)
0811 11:10:20.947061 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.947063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.947073 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:20.947104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.947105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.947115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.947174 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.947369 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.948142 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.949637 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.950232 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.950315 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.950325 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.950513 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.950555 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:10:20.950627 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.947103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:38189 (local address 127.29.145.3:44407)
0811 11:10:20.947152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.947154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.947335 (+ 181us) server_negotiation.cc:407] Connection header received
0811 11:10:20.947369 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.947370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.947379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.947438 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:20.947568 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.948250 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.949637 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.950239 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.950315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.950325 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.950513 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.950555 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:20.950640 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.946905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39539 (local address 127.29.145.2:45633)
0811 11:10:20.946950 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.946963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.947044 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.947201 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.947202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.947205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.947285 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.947287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.948231 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.948232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.949542 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.949544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.950323 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.950324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.950422 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.950424 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.950485 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.950544 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":86}
I20260811 11:10:20.950649 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.947033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:44407 (local address 127.29.145.2:38189)
0811 11:10:20.947239 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.947250 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.947312 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.947452 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.947452 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.947454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:20.947505 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.947506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.948330 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.948331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.949532 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.949536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.950344 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.950345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.950417 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.950422 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.950485 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.950544 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":8,"thread_start_us":122,"threads_started":1}
I20260811 11:10:20.950999 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbefd36ff98448095f3dee8eeddcd89" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" is_pre_election: true
I20260811 11:10:20.951098 32390 raft_consensus.cc:2454] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:20.951076 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbefd36ff98448095f3dee8eeddcd89" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e" is_pre_election: true
I20260811 11:10:20.951290 32712 raft_consensus.cc:2454] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:20.951278 32471 leader_election.cc:302] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, be27f76659384b5bbb850ea85fb1fb98; no voters:
I20260811 11:10:20.951404 327 raft_consensus.cc:2781] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:20.951448 327 raft_consensus.cc:482] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:20.951481 327 raft_consensus.cc:3037] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.952170 327 raft_consensus.cc:504] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.952304 327 leader_election.cc:288] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Requested vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:20.952487 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbefd36ff98448095f3dee8eeddcd89" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
I20260811 11:10:20.952488 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbefd36ff98448095f3dee8eeddcd89" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e"
I20260811 11:10:20.952579 32390 raft_consensus.cc:3037] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.952574 32712 raft_consensus.cc:3037] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:20.953209 32390 raft_consensus.cc:2454] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:20.953292 32712 raft_consensus.cc:2454] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:20.953439 32471 leader_election.cc:302] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, be27f76659384b5bbb850ea85fb1fb98; no voters:
I20260811 11:10:20.953573 327 raft_consensus.cc:2781] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:20.953790 327 raft_consensus.cc:686] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [term 1 LEADER]: Becoming Leader. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Running, Role: LEADER
I20260811 11:10:20.953907 327 consensus_queue.cc:237] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:20.954381 343 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: TxnStatusTablet state changed. Reason: New leader be27f76659384b5bbb850ea85fb1fb98. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.954493 343 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:20.954766 346 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:10:20.954798 32212 catalog_manager.cc:5495] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 reported cstate change: term changed from 0 to 1, leader changed from <none> to be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:20.954898 346 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:10:20.970642 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.968070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.29.145.2:34719)
0811 11:10:20.968164 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.968167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.968177 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:20.968271 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.968272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.968308 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.968372 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:20.968500 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.969036 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.969783 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.970010 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.970089 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.970277 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.970381 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.970469 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.970524 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.970545 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:10:20.970652 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.968000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34719 (local address 127.0.0.1:55684)
0811 11:10:20.968100 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.968113 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.968167 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.968378 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.968379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.968381 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:20.968435 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.968438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.969128 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.969129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.969697 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.969699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.970088 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.970089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.970157 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.970177 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.970447 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.970457 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.970517 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.970568 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 11:10:20.972811 32390 raft_consensus.cc:1261] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:20.972852 32712 raft_consensus.cc:1261] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:20.973150 327 consensus_queue.cc:1035] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:20.973291 343 consensus_queue.cc:1035] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:20.974459 328 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: TxnStatusTablet state changed. Reason: New leader be27f76659384b5bbb850ea85fb1fb98. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.974560 328 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.974491 329 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: TxnStatusTablet state changed. Reason: New leader be27f76659384b5bbb850ea85fb1fb98. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.974591 329 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.975991 327 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.976082 327 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:20.976183 327 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.976243 327 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P be27f76659384b5bbb850ea85fb1fb98: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:20.976243 351 mvcc.cc:205] Tried to move back new op lower bound from 7317285359501959168 to 7317285359427727360. Current Snapshot: MvccSnapshot[applied={T|T < 7317285359501959168}]
I20260811 11:10:20.976305 349 mvcc.cc:205] Tried to move back new op lower bound from 7317285359501959168 to 7317285359427727360. Current Snapshot: MvccSnapshot[applied={T|T < 7317285359501959168}]
I20260811 11:10:20.976493 328 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.976523 350 mvcc.cc:205] Tried to move back new op lower bound from 7317285359501959168 to 7317285359427727360. Current Snapshot: MvccSnapshot[applied={T|T < 7317285359501959168}]
I20260811 11:10:20.976567 328 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P c638d309513c4457a17e2f3ecab38d5e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.977034 329 tablet_replica.cc:439] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } } }
I20260811 11:10:20.977118 329 tablet_replica.cc:442] T 8bbefd36ff98448095f3dee8eeddcd89 P 9420a8ffea6d4380a8c96b0b94fd2da0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:20.982290 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.979684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.29.145.3:44407)
0811 11:10:20.979722 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.979724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.979732 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:20.979817 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.979818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.979831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.979881 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:20.980010 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.980454 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.981550 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.981735 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.981788 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.981980 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.982074 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.982134 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.982200 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.982217 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:10:20.987560 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.984881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34719 (local address 127.29.145.3:48669)
0811 11:10:20.984954 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.984966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.985011 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.985139 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.985140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.985141 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:20.985197 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.985198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.986100 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.986101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.986650 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.986652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.987114 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.987115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.987164 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.987177 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.987412 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.987415 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.987469 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.987510 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:10:20.987574 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.984962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:48669 (local address 127.29.145.2:34719)
0811 11:10:20.985008 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.985009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.985019 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:20.985064 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.985065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.985075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.985129 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:20.985261 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.986027 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.986739 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.987040 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.987109 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.987263 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.987296 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.987349 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.987486 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 11:10:20.987505 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:10:20.990072 32390 raft_consensus.cc:1261] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Refusing update from remote peer c638d309513c4457a17e2f3ecab38d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:20.990235 32552 raft_consensus.cc:1261] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Refusing update from remote peer c638d309513c4457a17e2f3ecab38d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:20.990337 328 consensus_queue.cc:1035] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:20.990497 331 consensus_queue.cc:1035] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:20.999770 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.996200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:44407 (local address 127.29.145.2:55135)
0811 11:10:20.996261 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:20.996273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:20.996320 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:20.996501 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:20.996502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:20.996504 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:20.996571 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.996572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.997148 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.997149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.998797 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:20.998800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.999289 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.999290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:20.999362 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.999381 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.999587 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.999589 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:20.999630 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 11:10:20.999685 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 11:10:21.000088 316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:20.996310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:55135 (local address 127.29.145.3:44407)
0811 11:10:20.996378 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:20.996380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:20.996387 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:10:20.996409 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:20.996410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:20.996423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:20.996478 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:20.996637 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.997077 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.998890 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:20.999178 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:20.999246 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:20.999479 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:20.999517 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:20.999562 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:20.999998 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 11:10:21.000030 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:21.010056 32211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:21.010176 32211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:21.011056 32211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Sending DeleteTablet for 3 replicas of tablet 6e3d73340aa145d9b60a75adebe23843
I20260811 11:10:21.011402 32528 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3d73340aa145d9b60a75adebe23843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:21 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55676
I20260811 11:10:21.011417 32690 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3d73340aa145d9b60a75adebe23843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:21 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55850
I20260811 11:10:21.011502 32370 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e3d73340aa145d9b60a75adebe23843 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:21 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:56280
I20260811 11:10:21.011874 372 tablet_replica.cc:330] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: stopping tablet replica
I20260811 11:10:21.012003 372 raft_consensus.cc:2229] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:21.012068 372 raft_consensus.cc:2258] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:21.012385 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:21.012531 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:21.012548 372 ts_tablet_manager.cc:1905] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:21.013962 32528 tablet_service.cc:1467] Processing CreateTablet for tablet 567d63289e4a4417bd000903620f46c8 (DEFAULT_TABLE table=test-table [id=f4962149e352411280d2831e15b71a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:21.014094 32528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567d63289e4a4417bd000903620f46c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:21.014987 372 ts_tablet_manager.cc:1918] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:21.015060 372 log.cc:1196] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/6e3d73340aa145d9b60a75adebe23843
I20260811 11:10:21.015340 372 ts_tablet_manager.cc:1939] T 6e3d73340aa145d9b60a75adebe23843 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting consensus metadata
I20260811 11:10:21.015432 321 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Bootstrap starting.
I20260811 11:10:21.015859 321 tablet_bootstrap.cc:654] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:21.016396 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 567d63289e4a4417bd000903620f46c8 (DEFAULT_TABLE table=test-table [id=f4962149e352411280d2831e15b71a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:21.016520 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567d63289e4a4417bd000903620f46c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:21.016572 321 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: No bootstrap required, opened a new log
I20260811 11:10:21.016629 321 ts_tablet_manager.cc:1397] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:21.016769 321 raft_consensus.cc:348] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.016840 321 raft_consensus.cc:374] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:21.016861 321 raft_consensus.cc:729] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Initialized, Role: FOLLOWER
I20260811 11:10:21.016908 321 consensus_queue.cc:260] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.016996 321 ts_tablet_manager.cc:1428] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:21.017897 322 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Bootstrap starting.
I20260811 11:10:21.018100 32192 catalog_manager.cc:4841] TS 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539): tablet 6e3d73340aa145d9b60a75adebe23843 (table test-table [id=f704868a005c4bda9ea657b4177b0dfe]) successfully deleted
I20260811 11:10:21.018350 322 tablet_bootstrap.cc:654] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:21.018338 32370 tablet_service.cc:1467] Processing CreateTablet for tablet 567d63289e4a4417bd000903620f46c8 (DEFAULT_TABLE table=test-table [id=f4962149e352411280d2831e15b71a6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:21.018446 32370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 567d63289e4a4417bd000903620f46c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:21.018996 322 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: No bootstrap required, opened a new log
I20260811 11:10:21.019048 322 ts_tablet_manager.cc:1397] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:21.019194 322 raft_consensus.cc:348] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.019264 322 raft_consensus.cc:374] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:21.019284 322 raft_consensus.cc:729] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Initialized, Role: FOLLOWER
I20260811 11:10:21.019328 322 consensus_queue.cc:260] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.019409 322 ts_tablet_manager.cc:1428] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:21.020117 323 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Bootstrap starting.
I20260811 11:10:21.020489 323 tablet_bootstrap.cc:654] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:21.021121 323 tablet_bootstrap.cc:492] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: No bootstrap required, opened a new log
I20260811 11:10:21.021175 323 ts_tablet_manager.cc:1397] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:21.021307 323 raft_consensus.cc:348] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.021375 323 raft_consensus.cc:374] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:21.021396 323 raft_consensus.cc:729] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420a8ffea6d4380a8c96b0b94fd2da0, State: Initialized, Role: FOLLOWER
I20260811 11:10:21.021441 323 consensus_queue.cc:260] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.021523 323 ts_tablet_manager.cc:1428] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:21.021677 371 tablet_replica.cc:330] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: stopping tablet replica
I20260811 11:10:21.021857 371 raft_consensus.cc:2229] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:21.022010 371 raft_consensus.cc:2258] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:21.022408 370 tablet_replica.cc:330] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: stopping tablet replica
I20260811 11:10:21.023016 371 ts_tablet_manager.cc:1905] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:21.024663 370 raft_consensus.cc:2229] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:21.024801 370 raft_consensus.cc:2258] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:21.025022 371 ts_tablet_manager.cc:1918] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:21.025108 371 log.cc:1196] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/6e3d73340aa145d9b60a75adebe23843
I20260811 11:10:21.025403 370 ts_tablet_manager.cc:1905] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:21.025424 371 ts_tablet_manager.cc:1939] T 6e3d73340aa145d9b60a75adebe23843 P c638d309513c4457a17e2f3ecab38d5e: Deleting consensus metadata
I20260811 11:10:21.025844 32192 catalog_manager.cc:4841] TS c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407): tablet 6e3d73340aa145d9b60a75adebe23843 (table test-table [id=f704868a005c4bda9ea657b4177b0dfe]) successfully deleted
I20260811 11:10:21.027170 370 ts_tablet_manager.cc:1918] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:21.027295 370 log.cc:1196] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/6e3d73340aa145d9b60a75adebe23843
I20260811 11:10:21.027608 370 ts_tablet_manager.cc:1939] T 6e3d73340aa145d9b60a75adebe23843 P be27f76659384b5bbb850ea85fb1fb98: Deleting consensus metadata
I20260811 11:10:21.028071 32192 catalog_manager.cc:4841] TS be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719): tablet 6e3d73340aa145d9b60a75adebe23843 (table test-table [id=f704868a005c4bda9ea657b4177b0dfe]) successfully deleted
I20260811 11:10:21.252767 331 raft_consensus.cc:482] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:21.252883 331 raft_consensus.cc:504] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.253077 331 leader_election.cc:288] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719)
I20260811 11:10:21.253450 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567d63289e4a4417bd000903620f46c8" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" is_pre_election: true
I20260811 11:10:21.253554 32390 raft_consensus.cc:2454] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 0.
I20260811 11:10:21.253571 32552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567d63289e4a4417bd000903620f46c8" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be27f76659384b5bbb850ea85fb1fb98" is_pre_election: true
I20260811 11:10:21.253702 32552 raft_consensus.cc:2454] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 0.
I20260811 11:10:21.253767 32635 leader_election.cc:302] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:21.253881 331 raft_consensus.cc:2781] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:21.253922 331 raft_consensus.cc:482] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:21.253952 331 raft_consensus.cc:3037] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:21.254685 331 raft_consensus.cc:504] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.254839 331 leader_election.cc:288] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 election: Requested vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719)
I20260811 11:10:21.255017 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567d63289e4a4417bd000903620f46c8" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
I20260811 11:10:21.255057 32552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "567d63289e4a4417bd000903620f46c8" candidate_uuid: "c638d309513c4457a17e2f3ecab38d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be27f76659384b5bbb850ea85fb1fb98"
I20260811 11:10:21.255112 32390 raft_consensus.cc:3037] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:21.255134 32552 raft_consensus.cc:3037] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:21.255975 32552 raft_consensus.cc:2454] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 1.
I20260811 11:10:21.255970 32390 raft_consensus.cc:2454] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c638d309513c4457a17e2f3ecab38d5e in term 1.
I20260811 11:10:21.256218 32635 leader_election.cc:302] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be27f76659384b5bbb850ea85fb1fb98, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:21.256309 331 raft_consensus.cc:2781] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:21.256384 331 raft_consensus.cc:686] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 1 LEADER]: Becoming Leader. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Running, Role: LEADER
I20260811 11:10:21.256457 331 consensus_queue.cc:237] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:21.257095 32211 catalog_manager.cc:5495] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to c638d309513c4457a17e2f3ecab38d5e (127.29.145.3). New cstate: current_term: 1 leader_uuid: "c638d309513c4457a17e2f3ecab38d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } health_report { overall_health: HEALTHY } } }
I20260811 11:10:21.311571 32390 raft_consensus.cc:1261] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Refusing update from remote peer c638d309513c4457a17e2f3ecab38d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:21.311571 32552 raft_consensus.cc:1261] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Refusing update from remote peer c638d309513c4457a17e2f3ecab38d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:21.311852 328 consensus_queue.cc:1035] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:21.311982 331 consensus_queue.cc:1035] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:10:21.316109 350 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 11:10:22.835786 32211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:22.835914 32211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:22.836872 32211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Sending DeleteTablet for 3 replicas of tablet 567d63289e4a4417bd000903620f46c8
I20260811 11:10:22.837212 32690 tablet_service.cc:1514] Processing DeleteTablet for tablet 567d63289e4a4417bd000903620f46c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55850
I20260811 11:10:22.837212 32370 tablet_service.cc:1514] Processing DeleteTablet for tablet 567d63289e4a4417bd000903620f46c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:56280
I20260811 11:10:22.837215 32528 tablet_service.cc:1514] Processing DeleteTablet for tablet 567d63289e4a4417bd000903620f46c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55676
I20260811 11:10:22.837453 408 tablet_replica.cc:330] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: stopping tablet replica
I20260811 11:10:22.837459 407 tablet_replica.cc:330] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: stopping tablet replica
I20260811 11:10:22.837706 408 raft_consensus.cc:2229] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:22.837770 408 raft_consensus.cc:2258] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.838064 407 raft_consensus.cc:2229] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:22.838184 407 raft_consensus.cc:2258] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.838172 408 ts_tablet_manager.cc:1905] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.838510 407 ts_tablet_manager.cc:1905] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.839947 408 ts_tablet_manager.cc:1918] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:10:22.840039 408 log.cc:1196] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/567d63289e4a4417bd000903620f46c8
I20260811 11:10:22.840344 408 ts_tablet_manager.cc:1939] T 567d63289e4a4417bd000903620f46c8 P be27f76659384b5bbb850ea85fb1fb98: Deleting consensus metadata
I20260811 11:10:22.840695 407 ts_tablet_manager.cc:1918] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:10:22.840758 407 log.cc:1196] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/567d63289e4a4417bd000903620f46c8
I20260811 11:10:22.840764 32192 catalog_manager.cc:4841] TS be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719): tablet 567d63289e4a4417bd000903620f46c8 (table test-table [id=f4962149e352411280d2831e15b71a6a]) successfully deleted
I20260811 11:10:22.841018 407 ts_tablet_manager.cc:1939] T 567d63289e4a4417bd000903620f46c8 P c638d309513c4457a17e2f3ecab38d5e: Deleting consensus metadata
I20260811 11:10:22.841161 406 tablet_replica.cc:330] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: stopping tablet replica
I20260811 11:10:22.841252 406 raft_consensus.cc:2229] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:22.841300 406 raft_consensus.cc:2258] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.841425 32192 catalog_manager.cc:4841] TS c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407): tablet 567d63289e4a4417bd000903620f46c8 (table test-table [id=f4962149e352411280d2831e15b71a6a]) successfully deleted
I20260811 11:10:22.841603 406 ts_tablet_manager.cc:1905] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.843077 406 ts_tablet_manager.cc:1918] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:10:22.843168 406 log.cc:1196] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/567d63289e4a4417bd000903620f46c8
I20260811 11:10:22.843389 406 ts_tablet_manager.cc:1939] T 567d63289e4a4417bd000903620f46c8 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting consensus metadata
I20260811 11:10:22.843789 32192 catalog_manager.cc:4841] TS 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539): tablet 567d63289e4a4417bd000903620f46c8 (table test-table [id=f4962149e352411280d2831e15b71a6a]) successfully deleted
Aug 11 11:10:22 dist-test-slave-69rg krb5kdc[32161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:10:22 dist-test-slave-69rg krb5kdc[32161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446622, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:22.867324 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:22.858982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.29.145.62:34527)
0811 11:10:22.859203 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:22.859205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:22.859215 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:22.859247 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:22.859248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:22.859261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:22.859327 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:22.859981 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.860638 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.861461 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.861699 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.861774 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:22.864320 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:22.864321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:22.864322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:22.864330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:22.865985 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:22.866318 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:22.866319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:22.866319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:22.866390 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:22.866648 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:22.866649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:22.866650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:22.866799 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:22.866906 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:22.867118 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 11:10:22.867237 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":139,"threads_started":1}
I20260811 11:10:22.869154 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:22.869381 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:22.870998 32370 tablet_service.cc:1467] Processing CreateTablet for tablet 9541c7f136734893b52dca8b50171979 (DEFAULT_TABLE table=test-table [id=f3a11aef73664f88831dd03af8bb59e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.871001 32528 tablet_service.cc:1467] Processing CreateTablet for tablet 9541c7f136734893b52dca8b50171979 (DEFAULT_TABLE table=test-table [id=f3a11aef73664f88831dd03af8bb59e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.871034 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 9541c7f136734893b52dca8b50171979 (DEFAULT_TABLE table=test-table [id=f3a11aef73664f88831dd03af8bb59e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.871192 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9541c7f136734893b52dca8b50171979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.871193 32528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9541c7f136734893b52dca8b50171979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.871197 32370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9541c7f136734893b52dca8b50171979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.872669 417 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Bootstrap starting.
I20260811 11:10:22.872933 418 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Bootstrap starting.
I20260811 11:10:22.873070 419 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Bootstrap starting.
I20260811 11:10:22.873126 417 tablet_bootstrap.cc:654] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.873437 418 tablet_bootstrap.cc:654] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.873550 419 tablet_bootstrap.cc:654] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.873888 417 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: No bootstrap required, opened a new log
I20260811 11:10:22.874078 417 ts_tablet_manager.cc:1397] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:22.874208 418 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: No bootstrap required, opened a new log
I20260811 11:10:22.874266 418 ts_tablet_manager.cc:1397] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:22.874408 418 raft_consensus.cc:348] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.874447 417 raft_consensus.cc:348] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.874498 418 raft_consensus.cc:374] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.874536 417 raft_consensus.cc:374] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.874536 418 raft_consensus.cc:729] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.874557 417 raft_consensus.cc:729] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420a8ffea6d4380a8c96b0b94fd2da0, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.874589 418 consensus_queue.cc:260] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.874629 417 consensus_queue.cc:260] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.874717 418 ts_tablet_manager.cc:1428] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:22.874832 417 ts_tablet_manager.cc:1428] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:22.875494 419 tablet_bootstrap.cc:492] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: No bootstrap required, opened a new log
I20260811 11:10:22.875584 419 ts_tablet_manager.cc:1397] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 11:10:22.875746 419 raft_consensus.cc:348] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.875834 419 raft_consensus.cc:374] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.875860 419 raft_consensus.cc:729] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.875921 419 consensus_queue.cc:260] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.876034 419 ts_tablet_manager.cc:1428] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:22.936378 356 raft_consensus.cc:482] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:22.936549 356 raft_consensus.cc:504] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.936738 356 leader_election.cc:288] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:22.937067 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9541c7f136734893b52dca8b50171979" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" is_pre_election: true
I20260811 11:10:22.937180 32390 raft_consensus.cc:2454] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:22.937096 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9541c7f136734893b52dca8b50171979" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e" is_pre_election: true
I20260811 11:10:22.937249 32712 raft_consensus.cc:2454] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:22.937408 32471 leader_election.cc:302] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, be27f76659384b5bbb850ea85fb1fb98; no voters:
I20260811 11:10:22.937533 356 raft_consensus.cc:2781] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:22.937583 356 raft_consensus.cc:482] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:22.937604 356 raft_consensus.cc:3037] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:22.938552 356 raft_consensus.cc:504] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.938696 356 leader_election.cc:288] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Requested vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:22.938879 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9541c7f136734893b52dca8b50171979" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
I20260811 11:10:22.938901 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9541c7f136734893b52dca8b50171979" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e"
I20260811 11:10:22.938973 32390 raft_consensus.cc:3037] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:22.938988 32712 raft_consensus.cc:3037] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:22.939698 32390 raft_consensus.cc:2454] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:22.939719 32712 raft_consensus.cc:2454] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:22.939898 32471 leader_election.cc:302] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be27f76659384b5bbb850ea85fb1fb98, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:22.940056 356 raft_consensus.cc:2781] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:22.940120 356 raft_consensus.cc:686] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 1 LEADER]: Becoming Leader. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Running, Role: LEADER
I20260811 11:10:22.940169 356 consensus_queue.cc:237] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.940999 32212 catalog_manager.cc:5495] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 reported cstate change: term changed from 0 to 1, leader changed from <none> to be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:22.952353 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:22.949934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.29.145.2:34719)
0811 11:10:22.950133 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:22.950135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:22.950144 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:22.950165 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:22.950166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:22.950178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:22.950259 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:22.950439 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.950971 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.951777 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.951982 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.952039 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:22.952156 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:22.952186 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:22.952221 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:22.952267 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 11:10:22.952285 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260811 11:10:22.957191 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:22.954949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:50225 (local address 127.29.145.2:34719)
0811 11:10:22.954993 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:22.954995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:22.955120 (+ 125us) server_negotiation.cc:407] Connection header received
0811 11:10:22.955171 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:22.955171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:22.955180 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:22.955224 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:22.955321 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.955787 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.956506 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.956679 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.956731 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:22.956931 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:22.956955 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:22.956994 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:22.957118 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 11:10:22.957136 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:10:22.957377 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:22.954895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:34719 (local address 127.29.145.2:50225)
0811 11:10:22.955061 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:22.955071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:22.955131 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:22.955235 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:22.955236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:22.955238 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:22.955280 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:22.955281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.955847 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.955848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:22.956432 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:22.956434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:22.956777 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:22.956778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:22.956837 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:22.956842 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:22.957013 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:22.957015 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:22.957113 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 11:10:22.957173 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260811 11:10:22.959640 32712 raft_consensus.cc:1261] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:22.959653 32390 raft_consensus.cc:1261] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:22.959861 356 consensus_queue.cc:1035] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:22.960000 382 consensus_queue.cc:1035] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:22.975251 32211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:22.975409 32211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:22.976192 32211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Sending DeleteTablet for 3 replicas of tablet 9541c7f136734893b52dca8b50171979
I20260811 11:10:22.976456 32370 tablet_service.cc:1514] Processing DeleteTablet for tablet 9541c7f136734893b52dca8b50171979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:56280
I20260811 11:10:22.976509 32690 tablet_service.cc:1514] Processing DeleteTablet for tablet 9541c7f136734893b52dca8b50171979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55850
I20260811 11:10:22.976576 406 tablet_replica.cc:330] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: stopping tablet replica
I20260811 11:10:22.976557 32528 tablet_service.cc:1514] Processing DeleteTablet for tablet 9541c7f136734893b52dca8b50171979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:22 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55676
I20260811 11:10:22.976619 407 tablet_replica.cc:330] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: stopping tablet replica
I20260811 11:10:22.976643 408 tablet_replica.cc:330] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: stopping tablet replica
I20260811 11:10:22.976636 406 raft_consensus.cc:2229] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:22.976675 407 raft_consensus.cc:2229] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:22.976697 408 raft_consensus.cc:2229] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:22.976711 406 raft_consensus.cc:2258] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.976713 407 raft_consensus.cc:2258] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.976796 408 raft_consensus.cc:2258] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:22.977196 406 ts_tablet_manager.cc:1905] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.977596 408 ts_tablet_manager.cc:1905] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.977742 407 ts_tablet_manager.cc:1905] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:22.979229 406 ts_tablet_manager.cc:1918] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:22.979288 406 log.cc:1196] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/9541c7f136734893b52dca8b50171979
I20260811 11:10:22.979436 408 ts_tablet_manager.cc:1918] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:22.979491 408 log.cc:1196] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/9541c7f136734893b52dca8b50171979
I20260811 11:10:22.979557 406 ts_tablet_manager.cc:1939] T 9541c7f136734893b52dca8b50171979 P 9420a8ffea6d4380a8c96b0b94fd2da0: Deleting consensus metadata
I20260811 11:10:22.979712 408 ts_tablet_manager.cc:1939] T 9541c7f136734893b52dca8b50171979 P be27f76659384b5bbb850ea85fb1fb98: Deleting consensus metadata
I20260811 11:10:22.979727 407 ts_tablet_manager.cc:1918] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:22.979784 407 log.cc:1196] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/9541c7f136734893b52dca8b50171979
I20260811 11:10:22.979938 32192 catalog_manager.cc:4841] TS 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539): tablet 9541c7f136734893b52dca8b50171979 (table test-table [id=f3a11aef73664f88831dd03af8bb59e0]) successfully deleted
I20260811 11:10:22.980031 32192 catalog_manager.cc:4841] TS be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719): tablet 9541c7f136734893b52dca8b50171979 (table test-table [id=f3a11aef73664f88831dd03af8bb59e0]) successfully deleted
I20260811 11:10:22.980031 407 ts_tablet_manager.cc:1939] T 9541c7f136734893b52dca8b50171979 P c638d309513c4457a17e2f3ecab38d5e: Deleting consensus metadata
I20260811 11:10:22.980140 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:22.980304 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:22.980408 32192 catalog_manager.cc:4841] TS c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407): tablet 9541c7f136734893b52dca8b50171979 (table test-table [id=f3a11aef73664f88831dd03af8bb59e0]) successfully deleted
I20260811 11:10:22.981803 32528 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 (DEFAULT_TABLE table=test-table [id=1138691d83e04153a52d0584d92b88e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.981861 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 (DEFAULT_TABLE table=test-table [id=1138691d83e04153a52d0584d92b88e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.981796 32370 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 (DEFAULT_TABLE table=test-table [id=1138691d83e04153a52d0584d92b88e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:22.981961 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0de89bc8514c868707a2d6d1af4004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.981961 32528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0de89bc8514c868707a2d6d1af4004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.982066 32370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0de89bc8514c868707a2d6d1af4004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:22.983279 418 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Bootstrap starting.
I20260811 11:10:22.983278 419 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Bootstrap starting.
I20260811 11:10:22.983654 418 tablet_bootstrap.cc:654] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.983654 419 tablet_bootstrap.cc:654] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.983657 417 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0: Bootstrap starting.
I20260811 11:10:22.984493 418 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: No bootstrap required, opened a new log
I20260811 11:10:22.984543 418 ts_tablet_manager.cc:1397] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:22.984560 417 tablet_bootstrap.cc:654] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:22.984669 419 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: No bootstrap required, opened a new log
I20260811 11:10:22.984712 419 ts_tablet_manager.cc:1397] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:22.984707 418 raft_consensus.cc:348] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.984789 418 raft_consensus.cc:374] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.984810 418 raft_consensus.cc:729] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c638d309513c4457a17e2f3ecab38d5e, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.984831 419 raft_consensus.cc:348] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.984859 418 consensus_queue.cc:260] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.984897 419 raft_consensus.cc:374] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.984920 419 raft_consensus.cc:729] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.984961 419 consensus_queue.cc:260] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.984979 418 ts_tablet_manager.cc:1428] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:22.985054 419 ts_tablet_manager.cc:1428] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:22.986249 417 tablet_bootstrap.cc:492] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0: No bootstrap required, opened a new log
I20260811 11:10:22.986299 417 ts_tablet_manager.cc:1397] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 11:10:22.986434 417 raft_consensus.cc:348] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.986496 417 raft_consensus.cc:374] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:22.986511 417 raft_consensus.cc:729] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420a8ffea6d4380a8c96b0b94fd2da0, State: Initialized, Role: FOLLOWER
I20260811 11:10:22.986544 417 consensus_queue.cc:260] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:22.986652 417 ts_tablet_manager.cc:1428] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:23.260262 389 raft_consensus.cc:482] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:23.260443 389 raft_consensus.cc:504] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:23.260711 389 leader_election.cc:288] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:23.261165 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0de89bc8514c868707a2d6d1af4004" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" is_pre_election: true
I20260811 11:10:23.261165 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0de89bc8514c868707a2d6d1af4004" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e" is_pre_election: true
I20260811 11:10:23.261312 32390 raft_consensus.cc:2454] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:23.261312 32712 raft_consensus.cc:2454] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 0.
I20260811 11:10:23.261548 32471 leader_election.cc:302] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9420a8ffea6d4380a8c96b0b94fd2da0, be27f76659384b5bbb850ea85fb1fb98; no voters:
I20260811 11:10:23.261685 389 raft_consensus.cc:2781] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:23.261737 389 raft_consensus.cc:482] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:23.261757 389 raft_consensus.cc:3037] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:23.262735 389 raft_consensus.cc:504] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:23.262895 389 leader_election.cc:288] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Requested vote from peers 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539), c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407)
I20260811 11:10:23.263109 32390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0de89bc8514c868707a2d6d1af4004" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0"
I20260811 11:10:23.263128 32712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0de89bc8514c868707a2d6d1af4004" candidate_uuid: "be27f76659384b5bbb850ea85fb1fb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c638d309513c4457a17e2f3ecab38d5e"
I20260811 11:10:23.263214 32390 raft_consensus.cc:3037] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:23.263218 32712 raft_consensus.cc:3037] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:23.264209 32390 raft_consensus.cc:2454] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:23.264209 32712 raft_consensus.cc:2454] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be27f76659384b5bbb850ea85fb1fb98 in term 1.
I20260811 11:10:23.264428 32471 leader_election.cc:302] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be27f76659384b5bbb850ea85fb1fb98, c638d309513c4457a17e2f3ecab38d5e; no voters:
I20260811 11:10:23.264559 389 raft_consensus.cc:2781] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:23.264618 389 raft_consensus.cc:686] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 1 LEADER]: Becoming Leader. State: Replica: be27f76659384b5bbb850ea85fb1fb98, State: Running, Role: LEADER
I20260811 11:10:23.264683 389 consensus_queue.cc:237] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } }
I20260811 11:10:23.265530 32211 catalog_manager.cc:5495] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 reported cstate change: term changed from 0 to 1, leader changed from <none> to be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "be27f76659384b5bbb850ea85fb1fb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be27f76659384b5bbb850ea85fb1fb98" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 34719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:23.276381 32390 raft_consensus.cc:1261] T 6c0de89bc8514c868707a2d6d1af4004 P 9420a8ffea6d4380a8c96b0b94fd2da0 [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:23.276424 32712 raft_consensus.cc:1261] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Refusing update from remote peer be27f76659384b5bbb850ea85fb1fb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:23.276636 382 consensus_queue.cc:1035] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9420a8ffea6d4380a8c96b0b94fd2da0" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:23.276762 389 consensus_queue.cc:1035] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c638d309513c4457a17e2f3ecab38d5e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:10:23.281862 380 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 11:10:24.806361 32211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:24.806555 32211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:24.808544 32211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 744764cb01d64c68b361ca4d423d87ac: Sending DeleteTablet for 3 replicas of tablet 6c0de89bc8514c868707a2d6d1af4004
I20260811 11:10:24.808843 32526 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55676
I20260811 11:10:24.808895 32690 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:55850
I20260811 11:10:24.808913 32370 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0de89bc8514c868707a2d6d1af4004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:56280
I20260811 11:10:24.809044 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 32294
I20260811 11:10:24.809145 465 tablet_replica.cc:330] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: stopping tablet replica
I20260811 11:10:24.814041 465 raft_consensus.cc:2229] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:24.814121 465 raft_consensus.cc:2258] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:24.814260 463 tablet_replica.cc:330] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: stopping tablet replica
I20260811 11:10:24.814332 463 raft_consensus.cc:2229] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:24.814420 463 raft_consensus.cc:2258] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:24.814642 465 ts_tablet_manager.cc:1905] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:24.815018 463 ts_tablet_manager.cc:1905] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:24.816795 465 ts_tablet_manager.cc:1918] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:10:24.816876 465 log.cc:1196] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/6c0de89bc8514c868707a2d6d1af4004
I20260811 11:10:24.817134 465 ts_tablet_manager.cc:1939] T 6c0de89bc8514c868707a2d6d1af4004 P c638d309513c4457a17e2f3ecab38d5e: Deleting consensus metadata
I20260811 11:10:24.817150 463 ts_tablet_manager.cc:1918] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:10:24.817203 463 log.cc:1196] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/6c0de89bc8514c868707a2d6d1af4004
I20260811 11:10:24.817495 463 ts_tablet_manager.cc:1939] T 6c0de89bc8514c868707a2d6d1af4004 P be27f76659384b5bbb850ea85fb1fb98: Deleting consensus metadata
I20260811 11:10:24.817637 32192 catalog_manager.cc:4841] TS c638d309513c4457a17e2f3ecab38d5e (127.29.145.3:44407): tablet 6c0de89bc8514c868707a2d6d1af4004 (table test-table [id=1138691d83e04153a52d0584d92b88e0]) successfully deleted
I20260811 11:10:24.817884 32192 catalog_manager.cc:4841] TS be27f76659384b5bbb850ea85fb1fb98 (127.29.145.2:34719): tablet 6c0de89bc8514c868707a2d6d1af4004 (table test-table [id=1138691d83e04153a52d0584d92b88e0]) successfully deleted
W20260811 11:10:24.822124 32192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:39539 (error 108)
I20260811 11:10:24.822564 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 32454
I20260811 11:10:24.824928 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:24.822748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39539 (local address 127.0.0.1:56296)
0811 11:10:24.824817 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:24.824875 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:39539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2037,"thread_start_us":76,"threads_started":1}
W20260811 11:10:24.825085 32192 catalog_manager.cc:4584] TS 9420a8ffea6d4380a8c96b0b94fd2da0 (127.29.145.1:39539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c0de89bc8514c868707a2d6d1af4004: Network error: Client connection negotiation failed: client connection to 127.29.145.1:39539: connect: Connection refused (error 111)
I20260811 11:10:24.833913 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 32615
I20260811 11:10:24.844520 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 32175
2026-08-11T11:10:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6160 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:24 dist-test-slave-69rg krb5kdc[472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:24 dist-test-slave-69rg krb5kdc[472](info): set up 2 sockets
Aug 11 11:10:24 dist-test-slave-69rg krb5kdc[472](info): commencing operation
krb5kdc: starting...
W20260811 11:10:26.018268 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:26.161423 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44549
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:45043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:44549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:26.246855 488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:26.247112 488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:26.247152 488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:26.248279 488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:26.248347 488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:26.248373 488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:26.248404 488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:26.248430 488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:26.249987 488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:45043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:44549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:26.250277 488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:26.250512 488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:26.255642 496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.255684 493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.255646 494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:26.255934 488 server_base.cc:1034] running on GCE node
I20260811 11:10:26.256224 488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:26.256569 488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:26.257774 488 hybrid_clock.cc:648] HybridClock initialized: now 1786446626257750 us; error 42 us; skew 500 ppm
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:26.260557 488 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:26.261014 488 webserver.cc:460] Webserver started at http://127.29.145.62:41173/ using document root <none> and password file <none>
I20260811 11:10:26.261270 488 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:26.261317 488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:26.261472 488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:26.262696 488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "a592882eacd0403cbb1c01ecfac3a3b0"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "ac9307d6fb023aa6c87f31f112ff3070"
server_key_iv: "80a4c55db2ad321c74b3c12660564a0e"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.263091 488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "a592882eacd0403cbb1c01ecfac3a3b0"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "ac9307d6fb023aa6c87f31f112ff3070"
server_key_iv: "80a4c55db2ad321c74b3c12660564a0e"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.264618 488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:26.265497 503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.265735 488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:26.265877 488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "a592882eacd0403cbb1c01ecfac3a3b0"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "ac9307d6fb023aa6c87f31f112ff3070"
server_key_iv: "80a4c55db2ad321c74b3c12660564a0e"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.265969 488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:26.294548 488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:26.297093 488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:26.297282 488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:26.301752 488 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:44549
I20260811 11:10:26.301836 561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:44549 every 8 connection(s)
I20260811 11:10:26.302225 488 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:26.303205 562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:26.305562 562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0: Bootstrap starting.
I20260811 11:10:26.306178 562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:26.306459 562 log.cc:824] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:26.307276 562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0: No bootstrap required, opened a new log
I20260811 11:10:26.307839 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 488
I20260811 11:10:26.307960 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:26.308153 30276 external_mini_cluster.cc:1428] Setting key 86b92dfcd128108ce2551bdb38d51a5a
I20260811 11:10:26.309540 562 raft_consensus.cc:348] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } }
I20260811 11:10:26.309743 562 raft_consensus.cc:374] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:26.309793 562 raft_consensus.cc:729] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a592882eacd0403cbb1c01ecfac3a3b0, State: Initialized, Role: FOLLOWER
I20260811 11:10:26.310004 562 consensus_queue.cc:260] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } }
I20260811 11:10:26.310110 562 raft_consensus.cc:388] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:26.310148 562 raft_consensus.cc:482] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:26.310202 562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:26.310998 562 raft_consensus.cc:504] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } }
I20260811 11:10:26.311138 562 leader_election.cc:302] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a592882eacd0403cbb1c01ecfac3a3b0; no voters:
I20260811 11:10:26.311357 562 leader_election.cc:288] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:26.311388 567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:26.311506 567 raft_consensus.cc:686] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [term 1 LEADER]: Becoming Leader. State: Replica: a592882eacd0403cbb1c01ecfac3a3b0, State: Running, Role: LEADER
I20260811 11:10:26.311666 562 sys_catalog.cc:564] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:26.311656 567 consensus_queue.cc:237] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } }
I20260811 11:10:26.312223 568 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } } }
I20260811 11:10:26.312408 568 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:26.312202 569 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a592882eacd0403cbb1c01ecfac3a3b0. Latest consensus state: current_term: 1 leader_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592882eacd0403cbb1c01ecfac3a3b0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44549 } } }
I20260811 11:10:26.312528 569 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:26.312987 574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:26.313537 574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:26.315210 574 catalog_manager.cc:1347] Generated new cluster ID: b78e72fa8aea4bb088c48c2794074aaa
I20260811 11:10:26.315268 574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:26.323963 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.308771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.29.145.62:44549)
0811 11:10:26.308934 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:26.308942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:26.308969 (+ 27us) server_negotiation.cc:407] Connection header received
0811 11:10:26.309695 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:26.309703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:26.309755 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:26.309949 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:26.310838 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.313896 (+ 3058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.315013 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.315621 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.315725 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.319004 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:26.319008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:26.319008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:26.319021 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:26.321781 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.322288 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.322291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.322292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.322405 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.322638 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.322639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.322639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.322953 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:26.323095 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:26.323403 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 11:10:26.323719 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260811 11:10:26.335115 574 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:26.335772 574 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:26.342257 574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a592882eacd0403cbb1c01ecfac3a3b0: Generated new TSK 0
I20260811 11:10:26.342552 574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:26.385581 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:45043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:26.476890 590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:26.477146 590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:26.477176 590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:26.478441 590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:26.478485 590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:26.478513 590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:26.480182 590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:45043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:26.480409 590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:26.480607 590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:26.486052 595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.486032 598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.486027 596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:26.486423 590 server_base.cc:1034] running on GCE node
I20260811 11:10:26.486660 590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:26.486928 590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:26.488075 590 hybrid_clock.cc:648] HybridClock initialized: now 1786446626488056 us; error 41 us; skew 500 ppm
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:26.490715 590 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:26.491225 590 webserver.cc:460] Webserver started at http://127.29.145.1:42453/ using document root <none> and password file <none>
I20260811 11:10:26.491477 590 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:26.491521 590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:26.491618 590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:26.492671 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "4fa94ab2b18a47c4845b54c18d27c77e"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "d5e4a8a21a7e61b2a225d6d2995b1d16"
server_key_iv: "ff3e0643dfdf218302dfeee4b18bbdbb"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.492981 590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "4fa94ab2b18a47c4845b54c18d27c77e"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "d5e4a8a21a7e61b2a225d6d2995b1d16"
server_key_iv: "ff3e0643dfdf218302dfeee4b18bbdbb"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.494331 590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:10:26.495148 605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.495375 590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:26.495476 590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "4fa94ab2b18a47c4845b54c18d27c77e"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "d5e4a8a21a7e61b2a225d6d2995b1d16"
server_key_iv: "ff3e0643dfdf218302dfeee4b18bbdbb"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.495530 590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:26.511951 590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:26.514379 590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:26.514506 590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:26.514721 590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:26.515069 590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:26.515105 590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.515129 590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:26.515165 590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.521070 590 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44545
I20260811 11:10:26.521159 733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44545 every 8 connection(s)
I20260811 11:10:26.521525 590 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:26.522015 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 590
I20260811 11:10:26.522130 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:26.522310 30276 external_mini_cluster.cc:1428] Setting key ffce828830544b98880ffcf8b371373c
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
I20260811 11:10:26.532860 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.522459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48431 (local address 127.29.145.62:44549)
0811 11:10:26.522533 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:26.522536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:26.522774 (+ 238us) server_negotiation.cc:407] Connection header received
0811 11:10:26.523497 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:26.523498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:26.523510 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:26.523566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:26.524937 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.525475 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.526648 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.526883 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.526967 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.529660 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:26.529661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:26.529661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:26.529669 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:26.531236 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.531765 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.531766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.531767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.531834 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.532113 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.532114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.532115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.532246 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:26.532324 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:26.532640 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 11:10:26.532749 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:10:26.533099 736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.522438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44549 (local address 127.29.145.1:48431)
0811 11:10:26.522637 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:26.522692 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:26.523368 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:26.523639 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:26.523644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:26.524519 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:26.524819 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.524826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.525602 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.525604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.526536 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.526539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.526982 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.526983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.527115 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.527731 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:26.527741 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:26.529510 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:26.531353 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.531364 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.531366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.531679 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.531905 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.531906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.531906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.532034 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.532328 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:26.532337 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:26.532494 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 11:10:26.532610 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:10:26.532832 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260811 11:10:26.533694 734 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44549
I20260811 11:10:26.533874 734 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:26.534147 734 heartbeater.cc:511] Master 127.29.145.62:44549 requested a full tablet report, sending...
I20260811 11:10:26.534830 520 ts_manager.cc:194] Registered new tserver with Master: 4fa94ab2b18a47c4845b54c18d27c77e (127.29.145.1:44545)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
I20260811 11:10:26.535548 520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:48431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:26.554850 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:45043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:26.641796 739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:26.642046 739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:26.642082 739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:26.643203 739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:26.643239 739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:26.643265 739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:26.644695 739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:45043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:26.644903 739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:26.645102 739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:26.650421 747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.650441 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:26.650732 739 server_base.cc:1034] running on GCE node
W20260811 11:10:26.650421 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:26.651063 739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:26.651376 739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:26.652559 739 hybrid_clock.cc:648] HybridClock initialized: now 1786446626652524 us; error 73 us; skew 500 ppm
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:26.655274 739 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:26.655755 739 webserver.cc:460] Webserver started at http://127.29.145.2:33111/ using document root <none> and password file <none>
I20260811 11:10:26.655988 739 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:26.656031 739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:26.656131 739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:26.657161 739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "289870372e4e4ccf90370ae283c90161"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "0343aee568d8e0d47b265b9b62202bcf"
server_key_iv: "2859d0623be92fcff81cf5453b0b9987"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.657467 739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "289870372e4e4ccf90370ae283c90161"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "0343aee568d8e0d47b265b9b62202bcf"
server_key_iv: "2859d0623be92fcff81cf5453b0b9987"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.658888 739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:26.659739 755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.659976 739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:26.660053 739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "289870372e4e4ccf90370ae283c90161"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "0343aee568d8e0d47b265b9b62202bcf"
server_key_iv: "2859d0623be92fcff81cf5453b0b9987"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.660184 739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:26.668334 739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:26.671087 739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:26.671276 739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:26.671566 739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:26.671911 739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:26.671947 739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.671972 739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:26.671984 739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.678352 739 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:33723
I20260811 11:10:26.678416 880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:33723 every 8 connection(s)
I20260811 11:10:26.678745 739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:26.681335 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 739
I20260811 11:10:26.681466 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:26.681669 30276 external_mini_cluster.cc:1428] Setting key 296984cf42f2cafe510c71b1480a01e5
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:26.689863 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.679561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:50145 (local address 127.29.145.62:44549)
0811 11:10:26.679625 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:26.679627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:26.679842 (+ 215us) server_negotiation.cc:407] Connection header received
0811 11:10:26.680392 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:26.680393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:26.680405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:26.680477 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:26.681669 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.682348 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.683615 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.683911 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.683987 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.686622 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:26.686623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:26.686624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:26.686632 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:26.688379 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.688784 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.688785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.688786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.688852 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.689100 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.689101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.689102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.689260 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:26.689343 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:26.689603 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 11:10:26.689693 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:10:26.689949 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.679510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44549 (local address 127.29.145.2:50145)
0811 11:10:26.679721 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:26.679763 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:26.680268 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:26.680601 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:26.680606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:26.681273 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:26.681550 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.681557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.682467 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.682468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.683518 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.683521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.684001 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.684002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.684109 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.684743 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:26.684756 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:26.686526 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:26.688464 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.688470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.688473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.688699 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.688930 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.688930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.688931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.689030 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.689340 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:26.689345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:26.689462 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 11:10:26.689550 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 11:10:26.689675 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":131,"threads_started":1}
I20260811 11:10:26.690537 881 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44549
I20260811 11:10:26.690659 881 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:26.690862 881 heartbeater.cc:511] Master 127.29.145.62:44549 requested a full tablet report, sending...
I20260811 11:10:26.691344 520 ts_manager.cc:194] Registered new tserver with Master: 289870372e4e4ccf90370ae283c90161 (127.29.145.2:33723)
I20260811 11:10:26.691887 520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:50145
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:26.713388 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:45043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:26.799461 887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:26.799665 887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:26.799690 887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:26.800833 887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:26.800873 887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:26.800899 887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:26.802275 887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:45043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:26.802454 887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:26.802631 887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:26.807704 894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.807644 896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:26.807648 892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:26.808092 887 server_base.cc:1034] running on GCE node
I20260811 11:10:26.808322 887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:26.808562 887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:26.809702 887 hybrid_clock.cc:648] HybridClock initialized: now 1786446626809689 us; error 34 us; skew 500 ppm
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:26.812428 887 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:26.812911 887 webserver.cc:460] Webserver started at http://127.29.145.3:33571/ using document root <none> and password file <none>
I20260811 11:10:26.813146 887 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:26.813223 887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:26.813365 887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:26.814522 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "cd9e545dffdd4a0dabc53aaca885fdbd"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "bbc3be22d133c4ebb320657a5a37344c"
server_key_iv: "3d821a348ba824983c210b5c2a35cd10"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.814898 887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "cd9e545dffdd4a0dabc53aaca885fdbd"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "bbc3be22d133c4ebb320657a5a37344c"
server_key_iv: "3d821a348ba824983c210b5c2a35cd10"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.816250 887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:26.816972 903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.817152 887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.817260 887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "cd9e545dffdd4a0dabc53aaca885fdbd"
format_stamp: "Formatted at 2026-08-11 11:10:26 on dist-test-slave-69rg"
server_key: "bbc3be22d133c4ebb320657a5a37344c"
server_key_iv: "3d821a348ba824983c210b5c2a35cd10"
server_key_version: "encryptionkey@0"
I20260811 11:10:26.817344 887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:26.829937 887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:26.831815 887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:26.831981 887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:26.832226 887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:26.832544 887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:26.832599 887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.832641 887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:26.832670 887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:26.838914 887 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:37749
I20260811 11:10:26.838960 1033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:37749 every 8 connection(s)
I20260811 11:10:26.839402 887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446626, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:26.848941 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.840221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:58187 (local address 127.29.145.62:44549)
0811 11:10:26.840289 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:26.840291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:26.840518 (+ 227us) server_negotiation.cc:407] Connection header received
0811 11:10:26.841022 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:26.841023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:26.841034 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:26.841093 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:26.842099 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.842689 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.843848 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.844038 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.844094 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.846448 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:26.846449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:26.846449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:26.846457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:26.847762 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.848161 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.848162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.848162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.848210 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:26.848404 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:26.848405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:26.848405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:26.848505 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:26.848556 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:26.848812 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 11:10:26.848873 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:10:26.849123 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:26.840187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44549 (local address 127.29.145.3:58187)
0811 11:10:26.840370 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:26.840413 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:26.840917 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:26.841148 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:26.841151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:26.841725 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:26.842005 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.842012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.842842 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.842843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.843751 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:26.843753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:26.844119 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:26.844119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:26.844263 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:26.844871 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:26.844881 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:26.846359 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:26.847858 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.847865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.847866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.848086 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.848262 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:26.848262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:26.848263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:26.848353 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:26.848564 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:26.848569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:26.848700 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 11:10:26.848786 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:10:26.848906 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260811 11:10:26.849507 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 887
I20260811 11:10:26.849599 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:26.849742 1034 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44549
I20260811 11:10:26.849855 30276 external_mini_cluster.cc:1428] Setting key 91e99408fb19eec1990a4f50701d1e66
I20260811 11:10:26.849864 1034 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:26.850062 1034 heartbeater.cc:511] Master 127.29.145.62:44549 requested a full tablet report, sending...
I20260811 11:10:26.850577 519 ts_manager.cc:194] Registered new tserver with Master: cd9e545dffdd4a0dabc53aaca885fdbd (127.29.145.3:37749)
I20260811 11:10:26.850975 519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:58187
I20260811 11:10:26.862275 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:10:26.877329 566 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:26.876311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.29.145.62:44549)
0811 11:10:26.876374 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:26.876376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:26.876499 (+ 123us) server_negotiation.cc:407] Connection header received
0811 11:10:26.876549 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:26.876550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:26.876560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:26.876640 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:26.877221 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54570: BlockingRecv error: recv got EOF from 127.0.0.1:54570 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:26.878309 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 590
I20260811 11:10:26.885691 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 739
I20260811 11:10:26.891644 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 887
I20260811 11:10:26.899238 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 488
2026-08-11T11:10:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2048 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[1048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[1048](info): set up 2 sockets
Aug 11 11:10:26 dist-test-slave-69rg krb5kdc[1048](info): commencing operation
krb5kdc: starting...
W20260811 11:10:28.026996 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:28.174643 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45889
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:45889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:28.259446 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:28.259670 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:28.259693 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:28.260946 1062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:28.260996 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:28.261010 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:28.261023 1062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:28.261035 1062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:28.262619 1062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:45889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:28.262899 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:28.263146 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:28.268316 1072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:28.268448 1062 server_base.cc:1034] running on GCE node
W20260811 11:10:28.268332 1069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:28.268316 1068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:28.268873 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:28.269125 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:28.270269 1062 hybrid_clock.cc:648] HybridClock initialized: now 1786446628270245 us; error 35 us; skew 500 ppm
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446628, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:28.272769 1062 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:28.273162 1062 webserver.cc:460] Webserver started at http://127.29.145.62:40397/ using document root <none> and password file <none>
I20260811 11:10:28.273401 1062 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:28.273471 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:28.273651 1062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:28.274852 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "4f60766d645c4c76a797275e0b84517d"
format_stamp: "Formatted at 2026-08-11 11:10:28 on dist-test-slave-69rg"
server_key: "f185d6ef94939a7bea926988b28b8485"
server_key_iv: "821f0937e3f27655cab9bed091f17994"
server_key_version: "encryptionkey@0"
I20260811 11:10:28.275169 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "4f60766d645c4c76a797275e0b84517d"
format_stamp: "Formatted at 2026-08-11 11:10:28 on dist-test-slave-69rg"
server_key: "f185d6ef94939a7bea926988b28b8485"
server_key_iv: "821f0937e3f27655cab9bed091f17994"
server_key_version: "encryptionkey@0"
I20260811 11:10:28.276510 1062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:28.277328 1081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:28.277588 1062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:28.277712 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "4f60766d645c4c76a797275e0b84517d"
format_stamp: "Formatted at 2026-08-11 11:10:28 on dist-test-slave-69rg"
server_key: "f185d6ef94939a7bea926988b28b8485"
server_key_iv: "821f0937e3f27655cab9bed091f17994"
server_key_version: "encryptionkey@0"
I20260811 11:10:28.277809 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:28.297933 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:28.300685 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:28.300890 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:28.305711 1062 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:45889
I20260811 11:10:28.305776 1138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:45889 every 8 connection(s)
I20260811 11:10:28.306455 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:28.307278 1139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:28.309638 1139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d: Bootstrap starting.
I20260811 11:10:28.310303 1139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:28.310561 1139 log.cc:824] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:28.310982 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1062
I20260811 11:10:28.311096 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:28.311291 30276 external_mini_cluster.cc:1428] Setting key dbaffcc5beb9b051c0b843a298a1aeaf
I20260811 11:10:28.311376 1139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d: No bootstrap required, opened a new log
I20260811 11:10:28.314756 1139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } }
I20260811 11:10:28.314947 1139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:28.315001 1139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f60766d645c4c76a797275e0b84517d, State: Initialized, Role: FOLLOWER
I20260811 11:10:28.315188 1139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } }
I20260811 11:10:28.315287 1139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:28.315321 1139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:28.315371 1139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:28.316191 1139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } }
I20260811 11:10:28.316322 1139 leader_election.cc:302] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f60766d645c4c76a797275e0b84517d; no voters:
I20260811 11:10:28.316500 1139 leader_election.cc:288] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:28.316541 1144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:28.316715 1144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [term 1 LEADER]: Becoming Leader. State: Replica: 4f60766d645c4c76a797275e0b84517d, State: Running, Role: LEADER
I20260811 11:10:28.316807 1139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:28.316861 1144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } }
I20260811 11:10:28.317252 1144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f60766d645c4c76a797275e0b84517d. Latest consensus state: current_term: 1 leader_uuid: "4f60766d645c4c76a797275e0b84517d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } } }
I20260811 11:10:28.317266 1145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f60766d645c4c76a797275e0b84517d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f60766d645c4c76a797275e0b84517d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45889 } } }
I20260811 11:10:28.317440 1144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:28.318512 1145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:28.318567 1162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:28.319188 1162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:28.321908 1162 catalog_manager.cc:1347] Generated new cluster ID: 337a8a7d5949432495f641795bb15b7a
I20260811 11:10:28.321985 1162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:28.323789 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:28.311856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.29.145.62:45889)
0811 11:10:28.312048 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:28.312057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:28.312084 (+ 27us) server_negotiation.cc:407] Connection header received
0811 11:10:28.312747 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:28.312756 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:28.312808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:28.312924 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:28.313932 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:28.314924 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:28.316072 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:28.316666 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:28.317130 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:28.319949 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:28.319954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:28.319954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:28.319966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:28.322068 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:28.322481 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:28.322484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:28.322485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:28.322608 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:28.322836 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:28.322837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:28.322838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:28.323166 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:28.323354 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:28.323426 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:10:28.323566 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260811 11:10:28.324851 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:10:28.333073 1162 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:28.333896 1162 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:28.342296 1162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f60766d645c4c76a797275e0b84517d: Generated new TSK 0
I20260811 11:10:28.342501 1162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:10:28.344142 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:28.340028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.29.145.62:45889)
0811 11:10:28.340091 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:28.340093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:28.340191 (+ 98us) server_negotiation.cc:407] Connection header received
0811 11:10:28.340244 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:28.340245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:28.340255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:28.340326 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:28.340742 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:28.341518 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:28.342677 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:28.342910 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:28.342971 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:28.343688 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:28.343690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:28.343690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:28.343874 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:28.343890 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:28.343896 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:28.343951 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:28.344033 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 11:10:28.344058 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:10:28.347306 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1062
2026-08-11T11:10:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1451 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1175](info): set up 2 sockets
Aug 11 11:10:28 dist-test-slave-69rg krb5kdc[1175](info): commencing operation
krb5kdc: starting...
W20260811 11:10:29.462560 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:29 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:29.595160 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45443
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:45443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:29.684540 1191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:29.684816 1191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:29.684863 1191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:29.686228 1191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:29.686290 1191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:29.686313 1191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:29.686342 1191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:29.686373 1191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:29.687919 1191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:45443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:29.688213 1191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:29.688457 1191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:29.693784 1198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:29.693794 1199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:29.693795 1202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:29.694032 1191 server_base.cc:1034] running on GCE node
I20260811 11:10:29.694375 1191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:29.694660 1191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:29.695813 1191 hybrid_clock.cc:648] HybridClock initialized: now 1786446629695788 us; error 36 us; skew 500 ppm
Aug 11 11:10:29 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446629, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:29.703447 1191 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:29.703929 1191 webserver.cc:460] Webserver started at http://127.29.145.62:45993/ using document root <none> and password file <none>
I20260811 11:10:29.704197 1191 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:29.704245 1191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:29.704406 1191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:29.705542 1191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "f8bf22662ccb4e439ffc92912a2ba873"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "e353b8504d456d939411ecbf033f9209"
server_key_iv: "62dc697bd76ad01d6e035e1df9af90e6"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.705948 1191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "f8bf22662ccb4e439ffc92912a2ba873"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "e353b8504d456d939411ecbf033f9209"
server_key_iv: "62dc697bd76ad01d6e035e1df9af90e6"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.707391 1191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:29.708230 1209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:29.708470 1191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:29.708585 1191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "f8bf22662ccb4e439ffc92912a2ba873"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "e353b8504d456d939411ecbf033f9209"
server_key_iv: "62dc697bd76ad01d6e035e1df9af90e6"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.708673 1191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:29.740635 1191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:29.743172 1191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:29.743372 1191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:29.748188 1191 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:45443
I20260811 11:10:29.748221 1266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:45443 every 8 connection(s)
I20260811 11:10:29.748620 1191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:29.749670 1267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:29.751739 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1191
I20260811 11:10:29.751864 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:29.752054 30276 external_mini_cluster.cc:1428] Setting key c979927a676f47b9be3bc6952915b823
I20260811 11:10:29.752238 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873: Bootstrap starting.
I20260811 11:10:29.753036 1267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:29.753350 1267 log.cc:824] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:29.754163 1267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873: No bootstrap required, opened a new log
I20260811 11:10:29.756558 1267 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } }
I20260811 11:10:29.756742 1267 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:29.756776 1267 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8bf22662ccb4e439ffc92912a2ba873, State: Initialized, Role: FOLLOWER
I20260811 11:10:29.756898 1267 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } }
I20260811 11:10:29.756989 1267 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:29.757019 1267 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:29.757057 1267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:29.757709 1267 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } }
I20260811 11:10:29.757853 1267 leader_election.cc:302] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8bf22662ccb4e439ffc92912a2ba873; no voters:
I20260811 11:10:29.758049 1267 leader_election.cc:288] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:29.758119 1272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:29.758277 1272 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [term 1 LEADER]: Becoming Leader. State: Replica: f8bf22662ccb4e439ffc92912a2ba873, State: Running, Role: LEADER
I20260811 11:10:29.758365 1267 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:29.758440 1272 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } }
I20260811 11:10:29.758847 1272 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8bf22662ccb4e439ffc92912a2ba873. Latest consensus state: current_term: 1 leader_uuid: "f8bf22662ccb4e439ffc92912a2ba873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } } }
I20260811 11:10:29.759090 1272 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:29.759008 1274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8bf22662ccb4e439ffc92912a2ba873" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8bf22662ccb4e439ffc92912a2ba873" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45443 } } }
I20260811 11:10:29.759536 1274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:29.759799 1284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 11:10:29 dist-test-slave-69rg krb5kdc[1175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:29.760327 1284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:29.762143 1284 catalog_manager.cc:1347] Generated new cluster ID: 80a0e4351ddb4419bdf84b130e9a0416
I20260811 11:10:29.762199 1284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:29.764389 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:29.752637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.29.145.62:45443)
0811 11:10:29.752830 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:29.752841 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:29.752873 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:10:29.753376 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:29.753383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:29.753431 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:29.753583 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:29.754365 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.755756 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.756546 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.757161 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.757279 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:29.760439 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:29.760443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:29.760443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:29.760456 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:29.762645 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:29.763058 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:29.763061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:29.763062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:29.763176 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:29.763393 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:29.763394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:29.763394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:29.763710 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:29.763885 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:29.764011 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 11:10:29.764167 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260811 11:10:29.770239 1284 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:29.770767 1284 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:29.774852 1284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8bf22662ccb4e439ffc92912a2ba873: Generated new TSK 0
I20260811 11:10:29.775056 1284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:29.805193 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:29.915180 1299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:29.915491 1299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:29.915534 1299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:29.916719 1299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:29.916761 1299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:29.916788 1299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:29.918296 1299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:29.918495 1299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:29.918685 1299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:29.924233 1307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:29.924219 1310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:29.924562 1299 server_base.cc:1034] running on GCE node
W20260811 11:10:29.924559 1306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:29.924840 1299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:29.925043 1299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:29.926189 1299 hybrid_clock.cc:648] HybridClock initialized: now 1786446629926178 us; error 33 us; skew 500 ppm
Aug 11 11:10:29 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446629, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:29.933388 1299 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:29.933955 1299 webserver.cc:460] Webserver started at http://127.29.145.1:40303/ using document root <none> and password file <none>
I20260811 11:10:29.934193 1299 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:29.934260 1299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:29.934401 1299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:29.935467 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "cf9d474b36494f978af48ef39311b085"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "fb7de87e192a9200e63b108ac2e49400"
server_key_iv: "ef45b4166bf1fa03f38d0f7bdf3fba9d"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.935851 1299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "cf9d474b36494f978af48ef39311b085"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "fb7de87e192a9200e63b108ac2e49400"
server_key_iv: "ef45b4166bf1fa03f38d0f7bdf3fba9d"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.937245 1299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:29.938036 1318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:29.938287 1299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:29.938405 1299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "cf9d474b36494f978af48ef39311b085"
format_stamp: "Formatted at 2026-08-11 11:10:29 on dist-test-slave-69rg"
server_key: "fb7de87e192a9200e63b108ac2e49400"
server_key_iv: "ef45b4166bf1fa03f38d0f7bdf3fba9d"
server_key_version: "encryptionkey@0"
I20260811 11:10:29.938480 1299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:29.953644 1299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:29.955972 1299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:29.956139 1299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:29.956387 1299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:29.956707 1299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:29.956760 1299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:29.956812 1299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:29.956859 1299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:29.963260 1299 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:45087
I20260811 11:10:29.963304 1443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:45087 every 8 connection(s)
I20260811 11:10:29.963656 1299 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:29 dist-test-slave-69rg krb5kdc[1175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446629, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:29.971709 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1299
I20260811 11:10:29.971832 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:29.972055 30276 external_mini_cluster.cc:1428] Setting key d157c2543300b82acc113aa0e8cebe2a
I20260811 11:10:29.973759 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:29.964566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:57447 (local address 127.29.145.62:45443)
0811 11:10:29.964671 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:29.964674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:29.964842 (+ 168us) server_negotiation.cc:407] Connection header received
0811 11:10:29.965401 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:29.965403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:29.965418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:29.965470 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:29.966542 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.967072 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.968120 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.968367 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.968446 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:29.970630 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:29.970632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:29.970633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:29.970641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:29.972248 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:29.972719 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:29.972721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:29.972721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:29.972795 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:29.973089 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:29.973090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:29.973090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:29.973193 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:29.973275 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:29.973613 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 11:10:29.973687 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 11:10:29.974071 1448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:29.964463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45443 (local address 127.29.145.1:57447)
0811 11:10:29.964694 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:29.964747 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:29.965276 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:29.965541 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:29.965550 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:29.966225 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:29.966431 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:29.966436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.967179 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.967180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:29.968025 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:29.968028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:29.968456 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:29.968458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:29.968580 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:29.969127 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:29.969135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:29.970527 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:29.972360 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:29.972368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:29.972370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:29.972643 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:29.972872 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:29.972873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:29.972873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:29.973018 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:29.973290 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:29.973297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:29.973462 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 11:10:29.973583 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 11:10:29.973765 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":113,"threads_started":1}
I20260811 11:10:29.974743 1445 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45443
I20260811 11:10:29.974858 1445 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:29.975077 1445 heartbeater.cc:511] Master 127.29.145.62:45443 requested a full tablet report, sending...
I20260811 11:10:29.975677 1227 ts_manager.cc:194] Registered new tserver with Master: cf9d474b36494f978af48ef39311b085 (127.29.145.1:45087)
I20260811 11:10:29.976585 1227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:57447
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:30.000526 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:30.089659 1451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:30.089916 1451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:30.089944 1451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:30.091048 1451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:30.091089 1451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:30.091120 1451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:30.092501 1451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:30.092689 1451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:30.092864 1451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:30.098006 1459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:30.098186 1456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:30.098022 1457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:30.098210 1451 server_base.cc:1034] running on GCE node
I20260811 11:10:30.098495 1451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:30.098687 1451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:30.099860 1451 hybrid_clock.cc:648] HybridClock initialized: now 1786446630099851 us; error 44 us; skew 500 ppm
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:30.102530 1451 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:30.103034 1451 webserver.cc:460] Webserver started at http://127.29.145.2:37837/ using document root <none> and password file <none>
I20260811 11:10:30.103327 1451 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:30.103397 1451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:30.103559 1451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:30.104631 1451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "4bf41a1738e5415883c4d7547650e05c"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "e557e32c9224383b524d6ce833edd1ba"
server_key_iv: "6f9d50b6d247f91d42265e2fbef2be45"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.105002 1451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "4bf41a1738e5415883c4d7547650e05c"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "e557e32c9224383b524d6ce833edd1ba"
server_key_iv: "6f9d50b6d247f91d42265e2fbef2be45"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.106544 1451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:30.107415 1467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.107640 1451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:30.107748 1451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "4bf41a1738e5415883c4d7547650e05c"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "e557e32c9224383b524d6ce833edd1ba"
server_key_iv: "6f9d50b6d247f91d42265e2fbef2be45"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.107836 1451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:30.126029 1451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:30.128262 1451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:30.128458 1451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:30.128736 1451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:30.129082 1451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:30.129145 1451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.129199 1451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:30.129226 1451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.136411 1451 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:34597
I20260811 11:10:30.136523 1597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:34597 every 8 connection(s)
I20260811 11:10:30.136879 1451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:30.146401 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1451
I20260811 11:10:30.146524 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:30.146766 30276 external_mini_cluster.cc:1428] Setting key cf7dc906b80e1211786746c219c7fb90
I20260811 11:10:30.147258 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:30.137747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41825 (local address 127.29.145.62:45443)
0811 11:10:30.137879 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:30.137881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:30.138024 (+ 143us) server_negotiation.cc:407] Connection header received
0811 11:10:30.138553 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:30.138554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:30.138565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:30.138631 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:30.139570 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.140068 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.141153 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.141386 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.141450 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:30.143992 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:30.143993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:30.143994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:30.144001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:30.145530 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.146029 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.146031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.146031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.146098 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.146404 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.146406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.146406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.146581 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:30.146662 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:30.147079 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 11:10:30.147192 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 11:10:30.147495 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:30.137696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45443 (local address 127.29.145.2:41825)
0811 11:10:30.137890 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:30.137933 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:30.138438 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:30.138687 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:30.138690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:30.139275 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:30.139471 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:30.139477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.140183 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.140183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:30.140988 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:30.140990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.141492 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.141494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:30.141625 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:30.142342 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:30.142351 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:30.143872 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:30.145620 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:30.145628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:30.145630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:30.145951 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:30.146206 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:30.146207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:30.146207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:30.146317 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:30.146681 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:30.146687 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:30.146879 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 11:10:30.147020 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 11:10:30.147234 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260811 11:10:30.148034 1598 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45443
I20260811 11:10:30.148190 1598 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:30.148413 1598 heartbeater.cc:511] Master 127.29.145.62:45443 requested a full tablet report, sending...
I20260811 11:10:30.148824 1227 ts_manager.cc:194] Registered new tserver with Master: 4bf41a1738e5415883c4d7547650e05c (127.29.145.2:34597)
I20260811 11:10:30.149261 1227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:41825
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:30.178367 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:30.263808 1604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:30.264063 1604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:30.264103 1604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:30.265314 1604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:30.265394 1604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:30.265442 1604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:30.267025 1604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:30.267355 1604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:30.267616 1604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:30.273411 1609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:30.273394 1612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:30.273394 1610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:30.273612 1604 server_base.cc:1034] running on GCE node
I20260811 11:10:30.273890 1604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:30.274195 1604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:30.275374 1604 hybrid_clock.cc:648] HybridClock initialized: now 1786446630275358 us; error 36 us; skew 500 ppm
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:30.278262 1604 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:30.278736 1604 webserver.cc:460] Webserver started at http://127.29.145.3:33087/ using document root <none> and password file <none>
I20260811 11:10:30.279042 1604 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:30.279093 1604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:30.279224 1604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:30.280323 1604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "9525068e6eaf4b3692b4027735bc145b"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "1da5f06bef66ea113cdf175f541df7ab"
server_key_iv: "12b1719e8ea0a201e0b0a2b9237f9d9f"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.280686 1604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "9525068e6eaf4b3692b4027735bc145b"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "1da5f06bef66ea113cdf175f541df7ab"
server_key_iv: "12b1719e8ea0a201e0b0a2b9237f9d9f"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.282191 1604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260811 11:10:30.283234 1620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.283461 1604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:30.283546 1604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "9525068e6eaf4b3692b4027735bc145b"
format_stamp: "Formatted at 2026-08-11 11:10:30 on dist-test-slave-69rg"
server_key: "1da5f06bef66ea113cdf175f541df7ab"
server_key_iv: "12b1719e8ea0a201e0b0a2b9237f9d9f"
server_key_version: "encryptionkey@0"
I20260811 11:10:30.283607 1604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:30.298786 1604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:30.300935 1604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:30.301059 1604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:30.301324 1604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:30.301678 1604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:30.301717 1604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.301740 1604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:30.301793 1604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:30.308245 1745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:43331 every 8 connection(s)
I20260811 11:10:30.308456 1604 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:43331
I20260811 11:10:30.308907 1604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:30.314783 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1604
I20260811 11:10:30.314894 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:30.315089 30276 external_mini_cluster.cc:1428] Setting key 378fda41c54cc03b16f53d757e37dd81
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:30.319711 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:30.309755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:50767 (local address 127.29.145.62:45443)
0811 11:10:30.309855 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:30.309857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:30.310073 (+ 216us) server_negotiation.cc:407] Connection header received
0811 11:10:30.310580 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:30.310581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:30.310592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:30.310638 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:30.311676 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.312247 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.313383 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.313589 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.313663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:30.316701 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:30.316702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:30.316703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:30.316711 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:30.318395 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.318815 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.318816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.318816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.318877 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.319106 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.319107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.319108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.319217 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:30.319269 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:30.319558 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 11:10:30.319640 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:10:30.319825 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:30.309726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45443 (local address 127.29.145.3:50767)
0811 11:10:30.309951 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:30.309990 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:30.310478 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:30.310733 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:30.310737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:30.311377 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:30.311570 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:30.311575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.312346 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.312347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:30.313284 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:30.313287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.313704 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.313706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:30.313892 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:30.314456 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:30.314465 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:30.316570 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:30.318488 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:30.318495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:30.318497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:30.318724 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:30.318947 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:30.318948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:30.318948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:30.319048 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:30.319276 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:30.319282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:30.319404 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 11:10:30.319494 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:10:30.319619 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":127,"threads_started":1}
I20260811 11:10:30.320466 1746 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45443
I20260811 11:10:30.320593 1746 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:30.320791 1746 heartbeater.cc:511] Master 127.29.145.62:45443 requested a full tablet report, sending...
I20260811 11:10:30.321208 1227 ts_manager.cc:194] Registered new tserver with Master: 9525068e6eaf4b3692b4027735bc145b (127.29.145.3:43331)
I20260811 11:10:30.321617 1227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:50767
I20260811 11:10:30.327502 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446630, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:30.354975 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:30.346710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.29.145.62:45443)
0811 11:10:30.346789 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:30.346791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:30.346809 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:10:30.346917 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:30.346918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:30.346928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:30.346989 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:30.347681 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.348206 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.348987 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:30.349185 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:30.349263 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:30.352029 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:30.352031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:30.352031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:30.352041 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:30.353642 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.354103 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.354105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.354105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.354169 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:30.354382 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:30.354383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:30.354384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:30.354556 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:30.354668 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:30.354774 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 11:10:30.354864 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 11:10:30.355257 1227 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58584
I20260811 11:10:30.355898 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1299
I20260811 11:10:30.366166 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1451
I20260811 11:10:30.380669 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1604
I20260811 11:10:30.388677 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1191
2026-08-11T11:10:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2038 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1761](info): set up 2 sockets
Aug 11 11:10:30 dist-test-slave-69rg krb5kdc[1761](info): commencing operation
krb5kdc: starting...
W20260811 11:10:31.546808 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:31 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:31.678803 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39909
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:31.763872 1776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:31.764101 1776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:31.764123 1776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:31.765223 1776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:31.765257 1776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:31.765271 1776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:31.765285 1776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:31.765296 1776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:31.766796 1776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:31.767005 1776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:31.767184 1776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:31.772395 1784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:31.772415 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:31.772437 1782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:31.772850 1776 server_base.cc:1034] running on GCE node
I20260811 11:10:31.773079 1776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:31.773347 1776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:31.774492 1776 hybrid_clock.cc:648] HybridClock initialized: now 1786446631774469 us; error 45 us; skew 500 ppm
Aug 11 11:10:31 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446631, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:31.777102 1776 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:31.777586 1776 webserver.cc:460] Webserver started at http://127.29.145.62:42967/ using document root <none> and password file <none>
I20260811 11:10:31.777896 1776 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:31.777971 1776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:31.778148 1776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:31.779263 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "b6cf9a06f8b5459dae4c46adceb5132c"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "21ba01e22b513a577745a2a41743ff10"
server_key_iv: "6bd1e76867293665f125a3d00ce9f62a"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.779645 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "b6cf9a06f8b5459dae4c46adceb5132c"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "21ba01e22b513a577745a2a41743ff10"
server_key_iv: "6bd1e76867293665f125a3d00ce9f62a"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.781141 1776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:31.782276 1792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:31.782526 1776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:31.782701 1776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "b6cf9a06f8b5459dae4c46adceb5132c"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "21ba01e22b513a577745a2a41743ff10"
server_key_iv: "6bd1e76867293665f125a3d00ce9f62a"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.782811 1776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:31.796315 1776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:31.798969 1776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:31.799103 1776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:31.803714 1776 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:39909
I20260811 11:10:31.803751 1847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:39909 every 8 connection(s)
I20260811 11:10:31.804329 1776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:31.804559 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1776
I20260811 11:10:31.804692 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:31.804908 30276 external_mini_cluster.cc:1428] Setting key 0b902bc8017b107d5d6f888e3d69d53a
I20260811 11:10:31.805220 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:31.808152 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c: Bootstrap starting.
I20260811 11:10:31.808815 1848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:31.809103 1848 log.cc:824] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:31.809792 1848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c: No bootstrap required, opened a new log
I20260811 11:10:31.811740 1848 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } }
Aug 11 11:10:31 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:31.811909 1848 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:31.811977 1848 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6cf9a06f8b5459dae4c46adceb5132c, State: Initialized, Role: FOLLOWER
I20260811 11:10:31.812142 1848 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } }
I20260811 11:10:31.812214 1848 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:31.812237 1848 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:31.812300 1848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:31.812892 1848 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } }
I20260811 11:10:31.813015 1848 leader_election.cc:302] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6cf9a06f8b5459dae4c46adceb5132c; no voters:
I20260811 11:10:31.813206 1848 leader_election.cc:288] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:31.813284 1853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:31.813445 1853 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [term 1 LEADER]: Becoming Leader. State: Replica: b6cf9a06f8b5459dae4c46adceb5132c, State: Running, Role: LEADER
I20260811 11:10:31.813491 1848 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:31.813885 1853 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } }
I20260811 11:10:31.814329 1856 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } } }
I20260811 11:10:31.814425 1856 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:31.814694 1864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:31.815331 1864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:31.815567 1855 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6cf9a06f8b5459dae4c46adceb5132c. Latest consensus state: current_term: 1 leader_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6cf9a06f8b5459dae4c46adceb5132c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39909 } } }
I20260811 11:10:31.815714 1855 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:31.816947 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:31.805392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.29.145.62:39909)
0811 11:10:31.805579 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:31.805590 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:31.805643 (+ 53us) server_negotiation.cc:407] Connection header received
0811 11:10:31.806378 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:31.806387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:31.806439 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:31.806598 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:31.807322 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.808351 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.809244 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.809689 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.809821 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:31.812539 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:31.812544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:31.812544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:31.812555 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:31.814770 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:31.815247 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:31.815250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:31.815250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:31.815378 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:31.815642 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:31.815643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:31.815643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:31.815983 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:31.816147 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:31.816383 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 11:10:31.816696 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260811 11:10:31.817435 1864 catalog_manager.cc:1347] Generated new cluster ID: 4514a6254d6340c2bf2caeab241d482d
I20260811 11:10:31.817495 1864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:31.827508 1864 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:31.828299 1864 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:31.837530 1864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6cf9a06f8b5459dae4c46adceb5132c: Generated new TSK 0
I20260811 11:10:31.837743 1864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:31.869987 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:31.956988 1878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:31.957208 1878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:31.957235 1878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:31.958461 1878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:31.958504 1878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:31.958534 1878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:31.960067 1878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:31.960335 1878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:31.960588 1878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:31.965898 1889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:31.965909 1885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:31.965955 1886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:31.966312 1878 server_base.cc:1034] running on GCE node
I20260811 11:10:31.966540 1878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:31.966787 1878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:31.967957 1878 hybrid_clock.cc:648] HybridClock initialized: now 1786446631967938 us; error 40 us; skew 500 ppm
Aug 11 11:10:31 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446631, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:31.970553 1878 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:31.971001 1878 webserver.cc:460] Webserver started at http://127.29.145.1:43743/ using document root <none> and password file <none>
I20260811 11:10:31.971254 1878 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:31.971326 1878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:31.971477 1878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:31.972515 1878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "f2d05bd8f6dc4601ad6b0893dcd57fe2"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "7dc0a70f8bc400402d785c7932151f6c"
server_key_iv: "aa5efe181324442e2beb70a3bb566e55"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.972885 1878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "f2d05bd8f6dc4601ad6b0893dcd57fe2"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "7dc0a70f8bc400402d785c7932151f6c"
server_key_iv: "aa5efe181324442e2beb70a3bb566e55"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.974299 1878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:31.975082 1900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:31.975261 1878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:31.975373 1878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "f2d05bd8f6dc4601ad6b0893dcd57fe2"
format_stamp: "Formatted at 2026-08-11 11:10:31 on dist-test-slave-69rg"
server_key: "7dc0a70f8bc400402d785c7932151f6c"
server_key_iv: "aa5efe181324442e2beb70a3bb566e55"
server_key_version: "encryptionkey@0"
I20260811 11:10:31.975449 1878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:31.983412 1878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:31.985180 1878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:31.985353 1878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:31.985671 1878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:31.986169 1878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:31.986228 1878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:31.986266 1878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:31.986299 1878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:31.992152 1878 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:33797
I20260811 11:10:31.992219 2027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:33797 every 8 connection(s)
I20260811 11:10:31.992537 1878 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:31.996416 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 1878
I20260811 11:10:31.996529 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:31.996711 30276 external_mini_cluster.cc:1428] Setting key 57ea8d25a1ee2a6a07527653183f3546
Aug 11 11:10:31 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446631, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:32.003213 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:31.993316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:59553 (local address 127.29.145.62:39909)
0811 11:10:31.993409 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:31.993412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:31.993501 (+ 89us) server_negotiation.cc:407] Connection header received
0811 11:10:31.994498 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:31.994499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:31.994511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:31.994576 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:31.995146 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.995660 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.996940 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.997263 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.997333 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:31.999964 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:31.999967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:31.999967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:31.999976 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:32.001624 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.002152 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.002153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.002154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.002221 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.002504 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.002505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.002505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.002634 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:32.002709 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:32.003020 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 11:10:32.003118 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 11:10:32.003405 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:31.993244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39909 (local address 127.29.145.1:59553)
0811 11:10:31.993402 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:31.993437 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:31.994383 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:31.994628 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:31.994631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:31.994867 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:31.995048 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:31.995053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.995743 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.995743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:31.996824 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:31.996827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:31.997377 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:31.997379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:31.997543 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:31.998181 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:31.998192 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:31.999860 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:32.001742 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.001751 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.001754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.002052 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.002298 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.002299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.002300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.002424 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.002712 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:32.002720 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:32.002873 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 11:10:32.002983 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 11:10:32.003157 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260811 11:10:32.003968 2028 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39909
I20260811 11:10:32.004096 2028 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:32.004326 2028 heartbeater.cc:511] Master 127.29.145.62:39909 requested a full tablet report, sending...
I20260811 11:10:32.004889 1811 ts_manager.cc:194] Registered new tserver with Master: f2d05bd8f6dc4601ad6b0893dcd57fe2 (127.29.145.1:33797)
I20260811 11:10:32.005520 1811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:59553
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:32.030373 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:32.120044 2033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:32.120272 2033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:32.120297 2033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:32.121438 2033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:32.121483 2033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:32.121511 2033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:32.123689 2033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:32.123930 2033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:32.124126 2033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:10:32.131278 2033 server_base.cc:1034] running on GCE node
W20260811 11:10:32.131374 2041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:32.131382 2038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:32.131873 2039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:32.132213 2033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:32.132555 2033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:32.133782 2033 hybrid_clock.cc:648] HybridClock initialized: now 1786446632133740 us; error 56 us; skew 500 ppm
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:32.136591 2033 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:32.137101 2033 webserver.cc:460] Webserver started at http://127.29.145.2:43069/ using document root <none> and password file <none>
I20260811 11:10:32.137351 2033 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:32.137420 2033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:32.137571 2033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:32.138713 2033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "573b69a53e1d4360ad21aaa6adc8c70f"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "deb0d5f89b280157cb96a3fb57a44d6d"
server_key_iv: "07b1ee30a839c29d68621e75fd93f8de"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.139101 2033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "573b69a53e1d4360ad21aaa6adc8c70f"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "deb0d5f89b280157cb96a3fb57a44d6d"
server_key_iv: "07b1ee30a839c29d68621e75fd93f8de"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.140460 2033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:32.141242 2050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.141424 2033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:32.141530 2033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "573b69a53e1d4360ad21aaa6adc8c70f"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "deb0d5f89b280157cb96a3fb57a44d6d"
server_key_iv: "07b1ee30a839c29d68621e75fd93f8de"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.141614 2033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:32.191100 2033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:32.193162 2033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:32.193339 2033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:32.193605 2033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:32.194005 2033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:32.194070 2033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.194113 2033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:32.194140 2033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.199995 2033 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:41727
I20260811 11:10:32.200040 2180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:41727 every 8 connection(s)
I20260811 11:10:32.200361 2033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:32.206965 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2033
I20260811 11:10:32.207079 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:32.207324 30276 external_mini_cluster.cc:1428] Setting key f49affd2b1022b7de1bc89d17d8e6747
I20260811 11:10:32.210368 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.201107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:50905 (local address 127.29.145.62:39909)
0811 11:10:32.201201 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:32.201204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:32.201380 (+ 176us) server_negotiation.cc:407] Connection header received
0811 11:10:32.202455 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:32.202457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:32.202471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:32.202526 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:32.203175 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.203683 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.204674 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.204886 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.204963 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.207109 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:32.207112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:32.207112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:32.207120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:32.208835 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.209275 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.209277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.209277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.209343 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.209636 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.209638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.209639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.209779 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:32.209909 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:32.210232 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 11:10:32.210306 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:10:32.210647 2183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.201066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39909 (local address 127.29.145.2:50905)
0811 11:10:32.201261 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:32.201298 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:32.202337 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:32.202601 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:32.202605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:32.202860 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:32.203048 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:32.203053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.203791 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.203793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:32.204568 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:32.204570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.204972 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.204973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:32.205102 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.205531 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:32.205540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:32.206981 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:32.208944 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.208952 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.208954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.209193 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.209412 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.209413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.209414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.209550 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.209902 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:32.209908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:32.210077 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 11:10:32.210205 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 11:10:32.210388 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260811 11:10:32.211158 2181 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39909
I20260811 11:10:32.211251 2181 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:32.211445 2181 heartbeater.cc:511] Master 127.29.145.62:39909 requested a full tablet report, sending...
I20260811 11:10:32.211820 1811 ts_manager.cc:194] Registered new tserver with Master: 573b69a53e1d4360ad21aaa6adc8c70f (127.29.145.2:41727)
I20260811 11:10:32.212219 1811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:50905
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:32.236443 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:32.321998 2186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:32.322268 2186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:32.322297 2186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:32.323782 2186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:32.323840 2186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:32.323873 2186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:32.325691 2186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:32.326026 2186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:32.326375 2186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:32.331995 2193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:32.332002 2195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:32.332038 2192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:32.332252 2186 server_base.cc:1034] running on GCE node
I20260811 11:10:32.332707 2186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:32.332958 2186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:32.334100 2186 hybrid_clock.cc:648] HybridClock initialized: now 1786446632334086 us; error 29 us; skew 500 ppm
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:32.336802 2186 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:32.337270 2186 webserver.cc:460] Webserver started at http://127.29.145.3:43613/ using document root <none> and password file <none>
I20260811 11:10:32.337601 2186 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:32.337678 2186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:32.337875 2186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:32.339046 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "072d99ba97594d3ca3923a54bf760861"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "c7481a48056e8f8ba9e8a7265af2b5d1"
server_key_iv: "8bc1b015463e5c4a43efb7936318c78b"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.339433 2186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "072d99ba97594d3ca3923a54bf760861"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "c7481a48056e8f8ba9e8a7265af2b5d1"
server_key_iv: "8bc1b015463e5c4a43efb7936318c78b"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.340838 2186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:32.341609 2202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.341797 2186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:32.341931 2186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "072d99ba97594d3ca3923a54bf760861"
format_stamp: "Formatted at 2026-08-11 11:10:32 on dist-test-slave-69rg"
server_key: "c7481a48056e8f8ba9e8a7265af2b5d1"
server_key_iv: "8bc1b015463e5c4a43efb7936318c78b"
server_key_version: "encryptionkey@0"
I20260811 11:10:32.342020 2186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:32.355232 2186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:32.358556 2186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:32.358763 2186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:32.359041 2186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:32.359745 2186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:32.359839 2186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.359882 2186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:32.359929 2186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:32.366021 2186 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:44125
I20260811 11:10:32.366104 2329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:44125 every 8 connection(s)
I20260811 11:10:32.366468 2186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:32.372710 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2186
I20260811 11:10:32.372813 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:32.373019 30276 external_mini_cluster.cc:1428] Setting key ed6230622f44a5a183c28d0c70d89ffb
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:32.376572 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.367231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:44853 (local address 127.29.145.62:39909)
0811 11:10:32.367328 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:32.367331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:32.367506 (+ 175us) server_negotiation.cc:407] Connection header received
0811 11:10:32.368471 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:32.368472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:32.368484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:32.368539 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:32.369129 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.369645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.370812 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.371006 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.371065 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.373701 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:32.373702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:32.373703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:32.373712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:32.375211 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.375672 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.375673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.375674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.375744 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.376004 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.376005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.376005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.376131 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:32.376206 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:32.376442 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 11:10:32.376508 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:10:32.376874 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.367192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39909 (local address 127.29.145.3:44853)
0811 11:10:32.367382 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:32.367418 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:32.368361 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:32.368594 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:32.368597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:32.368852 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:32.369045 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:32.369048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.369761 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.369762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:32.370710 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:32.370713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.371150 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.371151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:32.371267 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.371898 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:32.371908 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:32.373588 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:32.375308 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.375315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.375317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.375558 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.375817 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:32.375818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:32.375818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:32.375920 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:32.376214 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:32.376220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:32.376342 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 11:10:32.376432 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:10:32.376557 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260811 11:10:32.377458 2331 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39909
I20260811 11:10:32.377569 2331 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:32.377787 2331 heartbeater.cc:511] Master 127.29.145.62:39909 requested a full tablet report, sending...
I20260811 11:10:32.378261 1811 ts_manager.cc:194] Registered new tserver with Master: 072d99ba97594d3ca3923a54bf760861 (127.29.145.3:44125)
I20260811 11:10:32.378655 1811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:44853
I20260811 11:10:32.384939 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:32.409446 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.401770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.29.145.1:33797)
0811 11:10:32.401992 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:32.401998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:32.402010 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:32.402038 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:32.402040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:32.402055 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:32.402123 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:32.402703 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.403491 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.404205 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.404533 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.404600 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.406886 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:32.406894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:32.406894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:32.406901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:32.408386 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.408693 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.408696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.408696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.408760 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.408937 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.408937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.408938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.409062 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:32.409201 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:32.409235 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 11:10:32.409298 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":141,"threads_started":1}
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[1761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:32.418216 1851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:32.410844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.29.145.62:39909)
0811 11:10:32.410915 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:32.410917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:32.410952 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:10:32.411005 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:32.411006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:32.411015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:32.411061 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:32.411501 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.412031 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.412677 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:32.413020 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:32.413167 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:32.415398 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:32.415400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:32.415401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:32.415409 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:32.416923 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.417291 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.417292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.417293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.417354 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:32.417589 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:32.417590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:32.417591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:32.417747 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:32.417898 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:32.418002 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 11:10:32.418101 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:10:32.418467 1811 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47654
I20260811 11:10:32.418834 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1878
I20260811 11:10:32.428650 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2033
I20260811 11:10:32.436858 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2186
I20260811 11:10:32.443089 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 1776
2026-08-11T11:10:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2052 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[2345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[2345](info): set up 2 sockets
Aug 11 11:10:32 dist-test-slave-69rg krb5kdc[2345](info): commencing operation
krb5kdc: starting...
W20260811 11:10:33.567353 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:33 dist-test-slave-69rg krb5kdc[2345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:33.702278 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37251
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:33.786926 2360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:33.787228 2360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:33.787276 2360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:33.788524 2360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:33.788599 2360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:33.788623 2360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:33.788646 2360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:33.788676 2360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:33.790251 2360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:33.790529 2360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:33.790763 2360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:33.795845 2366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:33.795871 2365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:33.795845 2368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:33.796116 2360 server_base.cc:1034] running on GCE node
I20260811 11:10:33.796406 2360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:33.796728 2360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:33.797888 2360 hybrid_clock.cc:648] HybridClock initialized: now 1786446633797851 us; error 47 us; skew 500 ppm
Aug 11 11:10:33 dist-test-slave-69rg krb5kdc[2345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446633, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:33.800767 2360 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:33.801208 2360 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:33.801324 2360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:33.801486 2360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:33.802676 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "020cfbe5522047a185a4f9323cc7d33a"
format_stamp: "Formatted at 2026-08-11 11:10:33 on dist-test-slave-69rg"
server_key: "e3961eb448f995702aa207178cd5a77f"
server_key_iv: "d6a50b3c2fa8a1d5c307528cba1f66e0"
server_key_version: "encryptionkey@0"
I20260811 11:10:33.803061 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "020cfbe5522047a185a4f9323cc7d33a"
format_stamp: "Formatted at 2026-08-11 11:10:33 on dist-test-slave-69rg"
server_key: "e3961eb448f995702aa207178cd5a77f"
server_key_iv: "d6a50b3c2fa8a1d5c307528cba1f66e0"
server_key_version: "encryptionkey@0"
I20260811 11:10:33.804461 2360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:33.805331 2374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:33.805603 2360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:33.805755 2360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "020cfbe5522047a185a4f9323cc7d33a"
format_stamp: "Formatted at 2026-08-11 11:10:33 on dist-test-slave-69rg"
server_key: "e3961eb448f995702aa207178cd5a77f"
server_key_iv: "d6a50b3c2fa8a1d5c307528cba1f66e0"
server_key_version: "encryptionkey@0"
I20260811 11:10:33.805867 2360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:33.832114 2360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:33.834574 2360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:33.834756 2360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:33.839747 2360 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37251
I20260811 11:10:33.839813 2437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37251 every 8 connection(s)
I20260811 11:10:33.840103 2360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:33.841141 2438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:33.843709 2438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: Bootstrap starting.
I20260811 11:10:33.844360 2438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:33.844672 2438 log.cc:824] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:33.845505 2438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: No bootstrap required, opened a new log
I20260811 11:10:33.847159 2438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } }
I20260811 11:10:33.847368 2438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:33.847445 2438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020cfbe5522047a185a4f9323cc7d33a, State: Initialized, Role: FOLLOWER
I20260811 11:10:33.847617 2438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } }
I20260811 11:10:33.847707 2438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:33.847754 2438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:33.847803 2438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:33.848121 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2360
I20260811 11:10:33.848250 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:33.848480 30276 external_mini_cluster.cc:1428] Setting key c9bc349e62d3bf5a00882d3da6ff8d55
I20260811 11:10:33.849002 2438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } }
I20260811 11:10:33.849196 2438 leader_election.cc:302] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 020cfbe5522047a185a4f9323cc7d33a; no voters:
I20260811 11:10:33.849391 2438 leader_election.cc:288] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:33.849462 2443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:33.849624 2443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [term 1 LEADER]: Becoming Leader. State: Replica: 020cfbe5522047a185a4f9323cc7d33a, State: Running, Role: LEADER
I20260811 11:10:33.849686 2438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:33.850488 2443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } }
I20260811 11:10:33.851052 2447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "020cfbe5522047a185a4f9323cc7d33a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } } }
I20260811 11:10:33.851154 2447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:33.851588 2444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020cfbe5522047a185a4f9323cc7d33a. Latest consensus state: current_term: 1 leader_uuid: "020cfbe5522047a185a4f9323cc7d33a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020cfbe5522047a185a4f9323cc7d33a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37251 } } }
I20260811 11:10:33.851641 2461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:33.851675 2444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:33.852141 2461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:33.853615 2461 catalog_manager.cc:1347] Generated new cluster ID: 148ca4f5f2e140269609ab83f3c05f52
I20260811 11:10:33.853669 2461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:33 dist-test-slave-69rg krb5kdc[2345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:33.860752 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:33.849102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33436 (local address 127.29.145.62:37251)
0811 11:10:33.849326 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:33.849337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:33.849376 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:10:33.849508 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:33.849516 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:33.849580 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:33.850667 (+ 1087us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:33.850778 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:33.852242 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:33.853355 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:33.853976 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:33.854077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:33.856870 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:33.856874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:33.856875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:33.856888 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:33.859044 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:33.859444 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:33.859447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:33.859448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:33.859575 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:33.859793 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:33.859793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:33.859794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:33.860125 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:33.860311 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:33.860383 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:10:33.860532 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260811 11:10:33.879951 2461 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:33.880650 2461 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:33.889406 2461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: Generated new TSK 0
I20260811 11:10:33.889678 2461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:33.925395 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:34.014873 2468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:34.015075 2468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:34.015100 2468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:34.016206 2468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:34.016248 2468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:34.016274 2468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:34.018196 2468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:34.018489 2468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:34.018793 2468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:34.024168 2475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.024174 2479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.024174 2474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.024982 2468 server_base.cc:1034] running on GCE node
I20260811 11:10:34.025434 2468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:34.025895 2468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:34.027069 2468 hybrid_clock.cc:648] HybridClock initialized: now 1786446634027041 us; error 46 us; skew 500 ppm
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:34.029893 2468 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:34.030350 2468 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:34.030416 2468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:34.030514 2468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:34.031632 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "427b6e94adbe4be1b66d6efd2c127a80"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "367abde419fb76501642c0e2365239b9"
server_key_iv: "a1e6ba4f5d003d9a933758d7abe8f39e"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.031961 2468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "427b6e94adbe4be1b66d6efd2c127a80"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "367abde419fb76501642c0e2365239b9"
server_key_iv: "a1e6ba4f5d003d9a933758d7abe8f39e"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.033444 2468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:10:34.034621 2485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.034951 2468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:34.035104 2468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "427b6e94adbe4be1b66d6efd2c127a80"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "367abde419fb76501642c0e2365239b9"
server_key_iv: "a1e6ba4f5d003d9a933758d7abe8f39e"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.035233 2468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:34.045228 2468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:34.047894 2468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:34.048089 2468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:34.048367 2468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:34.048717 2468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:34.048775 2468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.048815 2468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:34.048866 2468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.056463 2468 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44159
I20260811 11:10:34.056573 2613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44159 every 8 connection(s)
I20260811 11:10:34.056878 2468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:34.061707 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2468
I20260811 11:10:34.061882 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:34.062135 30276 external_mini_cluster.cc:1428] Setting key 1c5097ce33d15c7a3c68eac81c781393
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:34.070000 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.057700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:43519 (local address 127.29.145.62:37251)
0811 11:10:34.057747 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.057749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.058029 (+ 280us) server_negotiation.cc:407] Connection header received
0811 11:10:34.058593 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.058595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.058612 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.058659 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:34.060053 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.061067 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.062830 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.063141 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.063213 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.066292 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:34.066294 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:34.066295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:34.066303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:34.068272 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.068819 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.068821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.068822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.068892 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.069155 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.069156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.069157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.069343 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:34.069419 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.069717 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.069851 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:34.070041 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.057678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37251 (local address 127.29.145.1:43519)
0811 11:10:34.057901 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.057944 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.058475 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.058745 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.058749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.059075 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:34.059953 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.059961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.061207 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.061208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.062734 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.062737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.063316 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.063317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.063492 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.064138 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:34.064149 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:34.066145 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:34.068367 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.068374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.068377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.068709 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.068966 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.068967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.068968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.069074 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.069417 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:34.069424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:34.069559 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.069645 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.069800 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260811 11:10:34.070734 2614 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37251
I20260811 11:10:34.070845 2614 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:34.071030 2614 heartbeater.cc:511] Master 127.29.145.62:37251 requested a full tablet report, sending...
I20260811 11:10:34.071630 2398 ts_manager.cc:194] Registered new tserver with Master: 427b6e94adbe4be1b66d6efd2c127a80 (127.29.145.1:44159)
I20260811 11:10:34.072274 2398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:43519
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:34.098650 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:34.187270 2619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:34.187497 2619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:34.187527 2619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:34.188838 2619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:34.188882 2619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:34.188910 2619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:34.190539 2619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:34.190815 2619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:34.191063 2619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:34.196736 2629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.196694 2627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.196856 2619 server_base.cc:1034] running on GCE node
W20260811 11:10:34.196693 2626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.197284 2619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:34.197540 2619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:34.198693 2619 hybrid_clock.cc:648] HybridClock initialized: now 1786446634198658 us; error 54 us; skew 500 ppm
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:34.201249 2619 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:34.201634 2619 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:34.201726 2619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:34.201926 2619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:34.202971 2619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "e4176b3916ee4066961680ba08cdbd2d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "98d5073f0bd1e8c821a414ca11d87024"
server_key_iv: "ebebee90099bfbe1f70bb9250fdf9b50"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.203346 2619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "e4176b3916ee4066961680ba08cdbd2d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "98d5073f0bd1e8c821a414ca11d87024"
server_key_iv: "ebebee90099bfbe1f70bb9250fdf9b50"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.204766 2619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:34.205513 2635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.205693 2619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:34.205792 2619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "e4176b3916ee4066961680ba08cdbd2d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "98d5073f0bd1e8c821a414ca11d87024"
server_key_iv: "ebebee90099bfbe1f70bb9250fdf9b50"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.205884 2619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:34.219350 2619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:34.221526 2619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:34.221698 2619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:34.222023 2619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:34.222384 2619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:34.222442 2619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.222482 2619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:34.222529 2619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.228464 2619 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:46185
I20260811 11:10:34.228549 2760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:46185 every 8 connection(s)
I20260811 11:10:34.228855 2619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:34.235111 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2619
I20260811 11:10:34.235240 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:34.235445 30276 external_mini_cluster.cc:1428] Setting key b2ff2d1521fbc2e20b8e3ee03bf25a0e
I20260811 11:10:34.239368 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.229774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:47745 (local address 127.29.145.62:37251)
0811 11:10:34.229903 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.229907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.230074 (+ 167us) server_negotiation.cc:407] Connection header received
0811 11:10:34.230683 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.230684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.230698 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.230752 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:34.231779 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.232305 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.233304 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.233528 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.233617 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.236005 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:34.236006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:34.236007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:34.236014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:34.237736 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.238199 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.238201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.238201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.238277 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.238624 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.238626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.238626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.238758 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:34.238811 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.239184 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.239261 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 11:10:34.239610 2764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.229682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37251 (local address 127.29.145.2:47745)
0811 11:10:34.229929 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.229975 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.230550 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.230824 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.230828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.231110 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:34.231672 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.231678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.232405 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.232406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.233209 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.233212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.233627 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.233629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.233763 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.234332 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:34.234341 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:34.235919 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:34.237866 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.237873 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.237875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.238119 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.238365 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.238367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.238367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.238522 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.238825 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:34.238831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:34.239004 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.239117 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.239329 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":134,"threads_started":1}
I20260811 11:10:34.240103 2761 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37251
I20260811 11:10:34.240199 2761 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:34.240367 2761 heartbeater.cc:511] Master 127.29.145.62:37251 requested a full tablet report, sending...
I20260811 11:10:34.240736 2398 ts_manager.cc:194] Registered new tserver with Master: e4176b3916ee4066961680ba08cdbd2d (127.29.145.2:46185)
I20260811 11:10:34.241106 2398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:47745
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:34.269760 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:34.359022 2768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:34.359243 2768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:34.359267 2768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:34.360354 2768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:34.360390 2768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:34.360416 2768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:34.361944 2768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:34.362180 2768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:34.362372 2768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:34.367594 2775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.367630 2778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.367702 2768 server_base.cc:1034] running on GCE node
W20260811 11:10:34.367601 2773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.368108 2768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:34.368374 2768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:34.369567 2768 hybrid_clock.cc:648] HybridClock initialized: now 1786446634369543 us; error 35 us; skew 500 ppm
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:34.372280 2768 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:34.372627 2768 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:34.372700 2768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:34.372859 2768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:34.374010 2768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "c20ef51d920743de9e8a00f45716f50e"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "3696063afdc53e11e6fae0e525cd15a6"
server_key_iv: "c5e3e4127d6270b8fd022ad435cfa9ce"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.374392 2768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "c20ef51d920743de9e8a00f45716f50e"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "3696063afdc53e11e6fae0e525cd15a6"
server_key_iv: "c5e3e4127d6270b8fd022ad435cfa9ce"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.375828 2768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:34.376695 2784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.376938 2768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:34.377053 2768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "c20ef51d920743de9e8a00f45716f50e"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "3696063afdc53e11e6fae0e525cd15a6"
server_key_iv: "c5e3e4127d6270b8fd022ad435cfa9ce"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.377131 2768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:34.392201 2768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:34.394773 2768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:34.394948 2768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:34.395221 2768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:34.395571 2768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:34.395627 2768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.395681 2768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:34.395710 2768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.401751 2768 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:38587
I20260811 11:10:34.401881 2911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:38587 every 8 connection(s)
I20260811 11:10:34.402141 2768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:34.406562 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2768
I20260811 11:10:34.406654 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:34.406855 30276 external_mini_cluster.cc:1428] Setting key 1cbc2c10d7ef143bccd0cacf0fe73f8c
Aug 11 11:10:34 dist-test-slave-69rg krb5kdc[2345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446634, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:34.412321 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.402994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:47041 (local address 127.29.145.62:37251)
0811 11:10:34.403044 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.403046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.403291 (+ 245us) server_negotiation.cc:407] Connection header received
0811 11:10:34.403873 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.403875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.403890 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.403942 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:34.404932 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.405576 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.406808 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.407076 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.407161 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.409540 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:34.409542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:34.409542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:34.409550 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:34.410897 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.411310 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.411311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.411312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.411371 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.411636 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.411637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.411637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.411761 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:34.411843 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.412129 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.412229 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:34.412496 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.402958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37251 (local address 127.29.145.3:47041)
0811 11:10:34.403152 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.403198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.403753 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.404015 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.404020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.404293 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:34.404823 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.404829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.405741 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.405743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.406657 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.406661 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.407167 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.407169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.407314 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.407873 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:34.407882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:34.409429 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:34.410985 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.410993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.410996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.411228 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.411451 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:34.411452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:34.411452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:34.411566 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:34.411832 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:34.411839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:34.411985 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.412090 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.412258 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260811 11:10:34.413003 2912 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37251
I20260811 11:10:34.413090 2912 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:34.413235 2912 heartbeater.cc:511] Master 127.29.145.62:37251 requested a full tablet report, sending...
I20260811 11:10:34.413597 2398 ts_manager.cc:194] Registered new tserver with Master: c20ef51d920743de9e8a00f45716f50e (127.29.145.3:38587)
I20260811 11:10:34.414068 2398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:47041
I20260811 11:10:34.418685 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:10:34.426970 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.420097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.29.145.62:37251)
0811 11:10:34.420188 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.420190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.420331 (+ 141us) server_negotiation.cc:407] Connection header received
0811 11:10:34.420357 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.420358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.420368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.420423 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:34.421123 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.421602 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.422699 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.423086 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.423164 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.424526 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:34.424527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:34.424528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:34.424535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:34.425902 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.426244 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.426245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.426246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.426307 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:34.426485 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:34.426485 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:34.426486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:34.426636 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:34.426717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.426811 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.426874 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:10:34.428818 2398 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:34.430608 2398 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:34.437675 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44159 (local address 127.0.0.1:38180)
0811 11:10:34.433691 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.433703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.433756 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.434092 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.434096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.434098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:34.434166 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.434168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435170 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.435172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.436380 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.436383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.437328 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.437329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.437416 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.437432 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.437529 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.437592 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260811 11:10:34.437700 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.29.145.1:44159)
0811 11:10:34.433742 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.433748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.433762 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:10:34.433868 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.433870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.433886 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.433970 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:34.434266 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435055 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.436488 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.437231 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.437328 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.437398 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.437563 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.437605 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260811 11:10:34.438234 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.29.145.3:38587)
0811 11:10:34.433905 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.433910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.433921 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:34.433966 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.433968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.433987 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.434326 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:34.434411 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435596 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.436887 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.437676 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.437907 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.437975 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.438109 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.438144 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":101,"threads_started":1}
I20260811 11:10:34.438418 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:38587 (local address 127.0.0.1:46790)
0811 11:10:34.433397 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.433442 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.433514 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.434119 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.434120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.434123 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:34.434235 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.434237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435743 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.435744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.436795 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.436797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.437933 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.437935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.438003 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.438007 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.438283 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.438334 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":25,"thread_start_us":63,"threads_started":1}
I20260811 11:10:34.438771 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:46185 (local address 127.0.0.1:57286)
0811 11:10:34.433690 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.433703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.433764 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.434203 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.434204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.434206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:34.434238 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.434240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435756 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.435758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.436894 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.436898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.438518 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.438519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.438596 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.438601 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.438666 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.438705 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":132,"threads_started":1}
I20260811 11:10:34.439069 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.433515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.29.145.2:46185)
0811 11:10:34.433950 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.433956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.433967 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:34.434006 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.434009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.434026 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.434106 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:34.434330 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.435597 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.436978 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.437845 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.437921 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.437975 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.438931 (+ 956us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.438984 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260811 11:10:34.439033 2536 tablet_service.cc:1467] Processing CreateTablet for tablet 68be4c277f0e4af692d1901b9de449b1 (DEFAULT_TABLE table=test-table [id=9c5f8bbae112457bb7e408887b61e460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:34.439409 2536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68be4c277f0e4af692d1901b9de449b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:34.439317 2836 tablet_service.cc:1467] Processing CreateTablet for tablet 68be4c277f0e4af692d1901b9de449b1 (DEFAULT_TABLE table=test-table [id=9c5f8bbae112457bb7e408887b61e460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:34.439606 2836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68be4c277f0e4af692d1901b9de449b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:34.442402 2929 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Bootstrap starting.
I20260811 11:10:34.442442 2930 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Bootstrap starting.
I20260811 11:10:34.443183 2930 tablet_bootstrap.cc:654] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:34.443183 2688 tablet_service.cc:1467] Processing CreateTablet for tablet 68be4c277f0e4af692d1901b9de449b1 (DEFAULT_TABLE table=test-table [id=9c5f8bbae112457bb7e408887b61e460]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:34.443532 2688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68be4c277f0e4af692d1901b9de449b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:34.443854 2930 log.cc:824] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:34.444684 2930 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: No bootstrap required, opened a new log
I20260811 11:10:34.444775 2930 ts_tablet_manager.cc:1397] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:34.446379 2932 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Bootstrap starting.
I20260811 11:10:34.446427 2930 raft_consensus.cc:348] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.446630 2930 raft_consensus.cc:374] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:34.446692 2930 raft_consensus.cc:729] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20ef51d920743de9e8a00f45716f50e, State: Initialized, Role: FOLLOWER
I20260811 11:10:34.446758 2929 tablet_bootstrap.cc:654] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:34.446838 2930 consensus_queue.cc:260] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.447193 2932 tablet_bootstrap.cc:654] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:34.447187 2930 ts_tablet_manager.cc:1428] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:10:34.447212 2912 heartbeater.cc:503] Master 127.29.145.62:37251 was elected leader, sending a full tablet report...
I20260811 11:10:34.447891 2932 log.cc:824] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:34.447893 2929 log.cc:824] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:34.448848 2929 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: No bootstrap required, opened a new log
I20260811 11:10:34.448971 2929 ts_tablet_manager.cc:1397] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 11:10:34.450268 2932 tablet_bootstrap.cc:492] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: No bootstrap required, opened a new log
I20260811 11:10:34.450356 2932 ts_tablet_manager.cc:1397] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:10:34.451166 2929 raft_consensus.cc:348] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.451349 2929 raft_consensus.cc:374] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:34.451390 2929 raft_consensus.cc:729] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 427b6e94adbe4be1b66d6efd2c127a80, State: Initialized, Role: FOLLOWER
I20260811 11:10:34.451515 2929 consensus_queue.cc:260] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.451851 2929 ts_tablet_manager.cc:1428] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:34.451860 2614 heartbeater.cc:503] Master 127.29.145.62:37251 was elected leader, sending a full tablet report...
I20260811 11:10:34.452483 2932 raft_consensus.cc:348] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.452672 2932 raft_consensus.cc:374] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:34.452703 2932 raft_consensus.cc:729] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4176b3916ee4066961680ba08cdbd2d, State: Initialized, Role: FOLLOWER
I20260811 11:10:34.452826 2932 consensus_queue.cc:260] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.453151 2932 ts_tablet_manager.cc:1428] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:34.453152 2761 heartbeater.cc:503] Master 127.29.145.62:37251 was elected leader, sending a full tablet report...
W20260811 11:10:34.479533 2763 tablet.cc:2369] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:34.557699 2615 tablet.cc:2369] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:34.644467 2938 raft_consensus.cc:482] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:34.644626 2938 raft_consensus.cc:504] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.645066 2938 leader_election.cc:288] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c20ef51d920743de9e8a00f45716f50e (127.29.145.3:38587), e4176b3916ee4066961680ba08cdbd2d (127.29.145.2:46185)
I20260811 11:10:34.649508 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.645408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45985 (local address 127.29.145.2:46185)
0811 11:10:34.645465 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.645467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.645665 (+ 198us) server_negotiation.cc:407] Connection header received
0811 11:10:34.645731 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.645732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.645741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.645794 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:34.646004 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.646769 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.648450 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.649051 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.649133 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.649143 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.649376 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.649443 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:10:34.649507 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.645407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:56407 (local address 127.29.145.3:38587)
0811 11:10:34.645475 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.645477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.645592 (+ 115us) server_negotiation.cc:407] Connection header received
0811 11:10:34.645705 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.645706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.645719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.645789 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:34.646009 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.646761 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.648451 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.649051 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.649133 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.649143 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.649377 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.649429 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:10:34.649534 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.645318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:46185 (local address 127.29.145.1:45985)
0811 11:10:34.645584 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.645594 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.645649 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.645829 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.645830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.645831 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:34.645929 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.645930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.646867 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.646868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.648334 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.648338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.649189 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.649190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.649251 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.649255 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.649309 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.649409 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":154,"thread_start_us":81,"threads_started":1}
I20260811 11:10:34.649537 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.645318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:38587 (local address 127.29.145.1:56407)
0811 11:10:34.645479 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.645491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:34.645589 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:34.645815 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:34.645816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:34.645818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:34.645911 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.645914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.646867 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.646868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.648334 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:34.648338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.649165 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.649167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:34.649237 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.649243 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:10:34.649311 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:10:34.649409 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":79,"thread_start_us":93,"threads_started":1}
I20260811 11:10:34.650106 2710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68be4c277f0e4af692d1901b9de449b1" candidate_uuid: "427b6e94adbe4be1b66d6efd2c127a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4176b3916ee4066961680ba08cdbd2d" is_pre_election: true
I20260811 11:10:34.650293 2710 raft_consensus.cc:2454] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 427b6e94adbe4be1b66d6efd2c127a80 in term 0.
I20260811 11:10:34.650472 2859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68be4c277f0e4af692d1901b9de449b1" candidate_uuid: "427b6e94adbe4be1b66d6efd2c127a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20ef51d920743de9e8a00f45716f50e" is_pre_election: true
I20260811 11:10:34.650570 2489 leader_election.cc:302] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 427b6e94adbe4be1b66d6efd2c127a80, e4176b3916ee4066961680ba08cdbd2d; no voters:
I20260811 11:10:34.650700 2859 raft_consensus.cc:2454] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 427b6e94adbe4be1b66d6efd2c127a80 in term 0.
I20260811 11:10:34.650765 2938 raft_consensus.cc:2781] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:34.650853 2938 raft_consensus.cc:482] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:34.650900 2938 raft_consensus.cc:3037] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:34.651808 2938 raft_consensus.cc:504] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.652045 2938 leader_election.cc:288] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [CANDIDATE]: Term 1 election: Requested vote from peers c20ef51d920743de9e8a00f45716f50e (127.29.145.3:38587), e4176b3916ee4066961680ba08cdbd2d (127.29.145.2:46185)
I20260811 11:10:34.652204 2710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68be4c277f0e4af692d1901b9de449b1" candidate_uuid: "427b6e94adbe4be1b66d6efd2c127a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4176b3916ee4066961680ba08cdbd2d"
I20260811 11:10:34.652313 2710 raft_consensus.cc:3037] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:34.652347 2859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68be4c277f0e4af692d1901b9de449b1" candidate_uuid: "427b6e94adbe4be1b66d6efd2c127a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c20ef51d920743de9e8a00f45716f50e"
I20260811 11:10:34.652439 2859 raft_consensus.cc:3037] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 0 FOLLOWER]: Advancing to term 1
W20260811 11:10:34.652776 2913 tablet.cc:2369] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:34.653242 2710 raft_consensus.cc:2454] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 427b6e94adbe4be1b66d6efd2c127a80 in term 1.
I20260811 11:10:34.653342 2859 raft_consensus.cc:2454] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 427b6e94adbe4be1b66d6efd2c127a80 in term 1.
I20260811 11:10:34.653471 2489 leader_election.cc:302] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 427b6e94adbe4be1b66d6efd2c127a80, e4176b3916ee4066961680ba08cdbd2d; no voters:
I20260811 11:10:34.653589 2938 raft_consensus.cc:2781] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:34.653836 2938 raft_consensus.cc:686] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [term 1 LEADER]: Becoming Leader. State: Replica: 427b6e94adbe4be1b66d6efd2c127a80, State: Running, Role: LEADER
I20260811 11:10:34.653960 2938 consensus_queue.cc:237] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } }
I20260811 11:10:34.654917 2397 catalog_manager.cc:5495] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 427b6e94adbe4be1b66d6efd2c127a80 (127.29.145.1). New cstate: current_term: 1 leader_uuid: "427b6e94adbe4be1b66d6efd2c127a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "427b6e94adbe4be1b66d6efd2c127a80" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:34.664889 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.662018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.29.145.1:44159)
0811 11:10:34.662086 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.662088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.662098 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:34.662158 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.662159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.662176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.662240 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:34.662525 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.663056 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.663876 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:34.664061 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:34.664128 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:34.664387 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:34.664488 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:34.664564 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.664778 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.664804 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:34.667349 2710 raft_consensus.cc:1261] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 1 FOLLOWER]: Refusing update from remote peer 427b6e94adbe4be1b66d6efd2c127a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:34.667451 2859 raft_consensus.cc:1261] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 1 FOLLOWER]: Refusing update from remote peer 427b6e94adbe4be1b66d6efd2c127a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:34.667896 2943 consensus_queue.cc:1035] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4176b3916ee4066961680ba08cdbd2d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:34.668051 2938 consensus_queue.cc:1035] T 68be4c277f0e4af692d1901b9de449b1 P 427b6e94adbe4be1b66d6efd2c127a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20ef51d920743de9e8a00f45716f50e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:34.670650 2946 mvcc.cc:205] Tried to move back new op lower bound from 7317285415593492480 to 7317285415543349248. Current Snapshot: MvccSnapshot[applied={T|T < 7317285415593492480}]
I20260811 11:10:34.670781 2949 mvcc.cc:205] Tried to move back new op lower bound from 7317285415593492480 to 7317285415543349248. Current Snapshot: MvccSnapshot[applied={T|T < 7317285415593492480}]
I20260811 11:10:34.671409 2947 mvcc.cc:205] Tried to move back new op lower bound from 7317285415593492480 to 7317285415543349248. Current Snapshot: MvccSnapshot[applied={T|T < 7317285415593492480}]
I20260811 11:10:34.673667 2398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:34.673776 2398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:34.674590 2398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 020cfbe5522047a185a4f9323cc7d33a: Sending DeleteTablet for 3 replicas of tablet 68be4c277f0e4af692d1901b9de449b1
I20260811 11:10:34.674980 2836 tablet_service.cc:1514] Processing DeleteTablet for tablet 68be4c277f0e4af692d1901b9de449b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:34 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46790
I20260811 11:10:34.675001 2688 tablet_service.cc:1514] Processing DeleteTablet for tablet 68be4c277f0e4af692d1901b9de449b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:34 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:57286
I20260811 11:10:34.675248 2957 tablet_replica.cc:330] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: stopping tablet replica
I20260811 11:10:34.675328 2536 tablet_service.cc:1514] Processing DeleteTablet for tablet 68be4c277f0e4af692d1901b9de449b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:34 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:38180
I20260811 11:10:34.675355 2957 raft_consensus.cc:2229] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:34.675352 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2468
I20260811 11:10:34.675455 2957 raft_consensus.cc:2258] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:34.675616 2959 tablet_replica.cc:330] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: stopping tablet replica
I20260811 11:10:34.675701 2959 raft_consensus.cc:2229] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:34.675788 2959 raft_consensus.cc:2258] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:34.676347 2959 ts_tablet_manager.cc:1905] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:34.678332 2957 ts_tablet_manager.cc:1905] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:34.678334 2959 ts_tablet_manager.cc:1918] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:34.678412 2959 log.cc:1196] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/68be4c277f0e4af692d1901b9de449b1
I20260811 11:10:34.678742 2959 ts_tablet_manager.cc:1939] T 68be4c277f0e4af692d1901b9de449b1 P c20ef51d920743de9e8a00f45716f50e: Deleting consensus metadata
I20260811 11:10:34.679177 2375 catalog_manager.cc:4841] TS c20ef51d920743de9e8a00f45716f50e (127.29.145.3:38587): tablet 68be4c277f0e4af692d1901b9de449b1 (table test-table [id=9c5f8bbae112457bb7e408887b61e460]) successfully deleted
I20260811 11:10:34.680357 2957 ts_tablet_manager.cc:1918] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:34.680455 2957 log.cc:1196] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/68be4c277f0e4af692d1901b9de449b1
I20260811 11:10:34.680758 2957 ts_tablet_manager.cc:1939] T 68be4c277f0e4af692d1901b9de449b1 P e4176b3916ee4066961680ba08cdbd2d: Deleting consensus metadata
I20260811 11:10:34.681229 2379 catalog_manager.cc:4841] TS e4176b3916ee4066961680ba08cdbd2d (127.29.145.2:46185): tablet 68be4c277f0e4af692d1901b9de449b1 (table test-table [id=9c5f8bbae112457bb7e408887b61e460]) successfully deleted
W20260811 11:10:34.683218 2377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:44159 (error 108)
I20260811 11:10:34.683513 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2619
I20260811 11:10:34.684829 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.683625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44159 (local address 127.0.0.1:38198)
0811 11:10:34.684723 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:34.684781 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1069}
W20260811 11:10:34.685091 2377 catalog_manager.cc:4584] TS 427b6e94adbe4be1b66d6efd2c127a80 (127.29.145.1:44159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68be4c277f0e4af692d1901b9de449b1: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44159: connect: Connection refused (error 111)
I20260811 11:10:34.694615 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2768
I20260811 11:10:34.701547 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2360
2026-08-11T11:10:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2257 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T11:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:34Z Disabled control of system clock
I20260811 11:10:34.732110 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44691
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:10:34.817448 2965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:34.817658 2965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:34.817682 2965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:34.819120 2965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:34.819204 2965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:34.819219 2965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:34.819232 2965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:34.819350 2965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:34.820910 2965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:44691
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:34.821214 2965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:34.821763 2965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:34.827204 2974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.827227 2971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.827315 2972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.827605 2965 server_base.cc:1034] running on GCE node
I20260811 11:10:34.827903 2965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:34.828253 2965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:34.829421 2965 hybrid_clock.cc:648] HybridClock initialized: now 1786446634829406 us; error 31 us; skew 500 ppm
I20260811 11:10:34.831036 2965 webserver.cc:460] Webserver started at http://127.29.145.62:43387/ using document root <none> and password file <none>
I20260811 11:10:34.831344 2965 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:34.831414 2965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:34.831568 2965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:34.833051 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "1b93d1daba154d86968618d7a5cb3f4d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "39aab0683d443170d7ddbb5d6bebb1b4"
server_key_iv: "0f43b24378546d223c513a43a5157375"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.833477 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "1b93d1daba154d86968618d7a5cb3f4d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "39aab0683d443170d7ddbb5d6bebb1b4"
server_key_iv: "0f43b24378546d223c513a43a5157375"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.835536 2965 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 11:10:34.836645 2980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:34.836889 2965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:34.836980 2965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "1b93d1daba154d86968618d7a5cb3f4d"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "39aab0683d443170d7ddbb5d6bebb1b4"
server_key_iv: "0f43b24378546d223c513a43a5157375"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.837075 2965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:34.841933 2965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:34.842233 2965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:34.842389 2965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:34.846452 2965 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:44691
I20260811 11:10:34.846488 3038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:44691 every 8 connection(s)
I20260811 11:10:34.846856 2965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:34.847581 3040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:34.850402 3040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: Bootstrap starting.
I20260811 11:10:34.850970 3040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:34.851228 3040 log.cc:824] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:34.851933 3040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: No bootstrap required, opened a new log
I20260811 11:10:34.853541 3040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } }
I20260811 11:10:34.853679 3040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:34.853701 3040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b93d1daba154d86968618d7a5cb3f4d, State: Initialized, Role: FOLLOWER
I20260811 11:10:34.853864 3040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } }
I20260811 11:10:34.853948 3040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:34.853972 3040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:34.853999 3040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:34.854514 3040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } }
I20260811 11:10:34.854598 3040 leader_election.cc:302] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b93d1daba154d86968618d7a5cb3f4d; no voters:
I20260811 11:10:34.854755 3040 leader_election.cc:288] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:34.854803 3043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:34.854929 3043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [term 1 LEADER]: Becoming Leader. State: Replica: 1b93d1daba154d86968618d7a5cb3f4d, State: Running, Role: LEADER
I20260811 11:10:34.855014 3043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } }
I20260811 11:10:34.855047 3040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:34.855497 3045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b93d1daba154d86968618d7a5cb3f4d. Latest consensus state: current_term: 1 leader_uuid: "1b93d1daba154d86968618d7a5cb3f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } } }
I20260811 11:10:34.855630 3045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:34.855798 3044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b93d1daba154d86968618d7a5cb3f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b93d1daba154d86968618d7a5cb3f4d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44691 } } }
I20260811 11:10:34.855907 3044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:34.856238 3050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:34.856771 3050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:34.857120 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 2965
I20260811 11:10:34.857235 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:34.857427 30276 external_mini_cluster.cc:1428] Setting key 13809a42176e1b5afdf7917741c19b9e
I20260811 11:10:34.858848 3050 catalog_manager.cc:1347] Generated new cluster ID: 242df4bb7d9f473097ee3c3ac04c992f
I20260811 11:10:34.858907 3050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:34.860216 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:34.858105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.29.145.62:44691)
0811 11:10:34.858358 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:34.858367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:34.858404 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:10:34.858522 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:34.858529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:34.858572 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:34.859053 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:34.859646 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:34.859649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:34.859649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:34.859653 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:34.859688 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:34.859714 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:34.859782 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:34.859875 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 11:10:34.860030 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":130,"threads_started":1}
I20260811 11:10:34.879938 3050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:34.880174 3050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:34.883368 3050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: Generated new TSK 0
I20260811 11:10:34.883592 3050 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:10:34.891084 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--builtin_ntp_servers=127.29.145.20:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:10:34.979645 3066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:34.979846 3066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:34.979868 3066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:34.981026 3066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:34.981074 3066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:34.981102 3066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:34.982546 3066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:34.982963 3066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:34.983296 3066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:34.989434 3072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.989490 3075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:34.989463 3071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:34.989892 3066 server_base.cc:1034] running on GCE node
I20260811 11:10:34.990118 3066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:34.990366 3066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:34.991547 3066 hybrid_clock.cc:648] HybridClock initialized: now 1786446634991521 us; error 43 us; skew 500 ppm
I20260811 11:10:34.996619 3066 webserver.cc:460] Webserver started at http://127.29.145.1:43139/ using document root <none> and password file <none>
I20260811 11:10:34.997037 3066 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:34.997143 3066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:34.997349 3066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:34.998876 3066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "15eaeed1475042469de5af384004d4d7"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "f8b521a818db040713b6f9b4c92d0968"
server_key_iv: "e2dab47d9a6e7afbe0700e3d31e83b2d"
server_key_version: "encryptionkey@0"
I20260811 11:10:34.999305 3066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "15eaeed1475042469de5af384004d4d7"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "f8b521a818db040713b6f9b4c92d0968"
server_key_iv: "e2dab47d9a6e7afbe0700e3d31e83b2d"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.001020 3066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:35.002120 3081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.002427 3066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:35.002571 3066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "15eaeed1475042469de5af384004d4d7"
format_stamp: "Formatted at 2026-08-11 11:10:34 on dist-test-slave-69rg"
server_key: "f8b521a818db040713b6f9b4c92d0968"
server_key_iv: "e2dab47d9a6e7afbe0700e3d31e83b2d"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.002665 3066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.007510 3066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.007982 3066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.008165 3066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.008708 3066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:35.009130 3066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:35.009195 3066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.009243 3066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:35.009306 3066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.017139 3066 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:46225
I20260811 11:10:35.017457 3221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:46225 every 8 connection(s)
I20260811 11:10:35.017653 3066 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:35.021332 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.018640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:42691 (local address 127.29.145.62:44691)
0811 11:10:35.018761 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.018763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.018970 (+ 207us) server_negotiation.cc:407] Connection header received
0811 11:10:35.019472 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.019475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.019486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.019556 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.020956 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.020959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.020959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.020965 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.020994 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.021000 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.021105 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.021247 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.021272 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 11:10:35.021476 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.018555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44691 (local address 127.29.145.1:42691)
0811 11:10:35.018756 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.018800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.019332 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.019675 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.019679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.019688 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.020752 (+ 1064us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.020753 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.020782 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.020813 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.021114 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.021119 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.021122 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.021191 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.021242 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260811 11:10:35.022367 3222 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44691
I20260811 11:10:35.022493 3222 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:35.022616 3222 heartbeater.cc:511] Master 127.29.145.62:44691 requested a full tablet report, sending...
I20260811 11:10:35.023478 3001 ts_manager.cc:194] Registered new tserver with Master: 15eaeed1475042469de5af384004d4d7 (127.29.145.1:46225)
I20260811 11:10:35.027868 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3066
I20260811 11:10:35.027959 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:35.028103 30276 external_mini_cluster.cc:1428] Setting key d29f0b8232f12e2d399cd39ee3072342
I20260811 11:10:35.029652 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--builtin_ntp_servers=127.29.145.20:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:10:35.116299 3225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:35.116515 3225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:35.116537 3225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:35.117766 3225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:35.117837 3225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:35.117866 3225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:35.119323 3225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:35.119580 3225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:35.119895 3225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:35.125214 3231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.125202 3234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.125206 3232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.125501 3225 server_base.cc:1034] running on GCE node
I20260811 11:10:35.125728 3225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:35.125990 3225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:35.127177 3225 hybrid_clock.cc:648] HybridClock initialized: now 1786446635127147 us; error 55 us; skew 500 ppm
I20260811 11:10:35.128466 3225 webserver.cc:460] Webserver started at http://127.29.145.2:34879/ using document root <none> and password file <none>
I20260811 11:10:35.128732 3225 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:35.128815 3225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:35.128979 3225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:35.130151 3225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "a49c9c3cc9664dbf904d7c0240bda07c"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "e0ed3c24bf2d6075d67c02d62340b55c"
server_key_iv: "bfbfc3083d740fe218976aaf9a0b0401"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.130520 3225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "a49c9c3cc9664dbf904d7c0240bda07c"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "e0ed3c24bf2d6075d67c02d62340b55c"
server_key_iv: "bfbfc3083d740fe218976aaf9a0b0401"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.131887 3225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:35.132681 3240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.132870 3225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:35.132978 3225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "a49c9c3cc9664dbf904d7c0240bda07c"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "e0ed3c24bf2d6075d67c02d62340b55c"
server_key_iv: "bfbfc3083d740fe218976aaf9a0b0401"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.133054 3225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.136440 3225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.136701 3225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.136873 3225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.137115 3225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:35.137436 3225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:35.137490 3225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.137529 3225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:35.137557 3225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.143756 3225 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:44287
I20260811 11:10:35.143839 3369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:44287 every 8 connection(s)
I20260811 11:10:35.144170 3225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:35.147228 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3225
I20260811 11:10:35.147269 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.144980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:37839 (local address 127.29.145.62:44691)
0811 11:10:35.145048 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.145051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.145235 (+ 184us) server_negotiation.cc:407] Connection header received
0811 11:10:35.145732 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.145733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.145740 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.145807 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.147005 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.147007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.147007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.147011 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.147028 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.147033 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.147089 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.147196 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.147209 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:10:35.147339 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:35.147512 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.144899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44691 (local address 127.29.145.2:37839)
0811 11:10:35.145100 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.145142 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.145616 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.145941 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.145945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.145954 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.146853 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.146854 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.146881 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.146907 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.147101 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.147107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.147111 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.147203 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.147270 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260811 11:10:35.147567 30276 external_mini_cluster.cc:1428] Setting key cac7160e95074a5ffc5628fc096a9f76
I20260811 11:10:35.148077 3370 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44691
I20260811 11:10:35.148178 3370 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:35.148327 3370 heartbeater.cc:511] Master 127.29.145.62:44691 requested a full tablet report, sending...
I20260811 11:10:35.148710 3000 ts_manager.cc:194] Registered new tserver with Master: a49c9c3cc9664dbf904d7c0240bda07c (127.29.145.2:44287)
I20260811 11:10:35.148998 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--builtin_ntp_servers=127.29.145.20:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:10:35.235401 3374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:35.235750 3374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:35.235778 3374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:35.237193 3374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:35.237237 3374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:35.237300 3374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:35.238933 3374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44691
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:35.239246 3374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:35.239605 3374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:35.245618 3380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.245724 3374 server_base.cc:1034] running on GCE node
W20260811 11:10:35.245622 3382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.245622 3379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.246209 3374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:35.246510 3374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:35.247732 3374 hybrid_clock.cc:648] HybridClock initialized: now 1786446635247720 us; error 40 us; skew 500 ppm
I20260811 11:10:35.249262 3374 webserver.cc:460] Webserver started at http://127.29.145.3:43283/ using document root <none> and password file <none>
I20260811 11:10:35.249554 3374 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:35.249647 3374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:35.249863 3374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:35.251015 3374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "3ff8c3edd7dd406eae6f50c1170ea792"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "3ae07eb6627512ee5ca37c13c81568e3"
server_key_iv: "d75fe32364390e9c82e64daba8bd66c8"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.251415 3374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "3ff8c3edd7dd406eae6f50c1170ea792"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "3ae07eb6627512ee5ca37c13c81568e3"
server_key_iv: "d75fe32364390e9c82e64daba8bd66c8"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.252851 3374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:35.253648 3388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.253883 3374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:35.253991 3374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "3ff8c3edd7dd406eae6f50c1170ea792"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "3ae07eb6627512ee5ca37c13c81568e3"
server_key_iv: "d75fe32364390e9c82e64daba8bd66c8"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.254081 3374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.257696 3374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.257987 3374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.258150 3374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.258374 3374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:35.258697 3374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:35.258757 3374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.258795 3374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:35.258823 3374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.264745 3374 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:46845
I20260811 11:10:35.264770 3521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:46845 every 8 connection(s)
I20260811 11:10:35.265105 3374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:35.268015 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.265905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:40121 (local address 127.29.145.62:44691)
0811 11:10:35.265974 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.265977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.266198 (+ 221us) server_negotiation.cc:407] Connection header received
0811 11:10:35.266681 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.266682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.266688 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.266738 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.267831 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.267832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.267832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.267836 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.267849 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.267853 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.267896 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.267948 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.267961 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:35.268230 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.265867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44691 (local address 127.29.145.3:40121)
0811 11:10:35.266078 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.266112 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.266594 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.266792 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.266795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.266804 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.267682 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.267683 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.267713 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.267739 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.267901 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.267903 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.267906 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.267954 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.267999 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":121,"threads_started":1}
I20260811 11:10:35.268735 3522 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44691
I20260811 11:10:35.268844 3522 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:35.269001 3522 heartbeater.cc:511] Master 127.29.145.62:44691 requested a full tablet report, sending...
I20260811 11:10:35.269377 3000 ts_manager.cc:194] Registered new tserver with Master: 3ff8c3edd7dd406eae6f50c1170ea792 (127.29.145.3:46845)
I20260811 11:10:35.274963 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3374
I20260811 11:10:35.275058 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:35.275195 30276 external_mini_cluster.cc:1428] Setting key 10ca549c485f38c476895639e23f42c9
I20260811 11:10:35.276008 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:10:35.278491 3064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.277359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.29.145.62:44691)
0811 11:10:35.277420 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.277422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.277586 (+ 164us) server_negotiation.cc:407] Connection header received
0811 11:10:35.277609 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.277610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.277615 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.277662 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.278255 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.278256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.278257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.278260 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.278276 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.278280 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.278331 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.278378 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.278389 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:10:35.280258 3000 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:35.281783 3000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:35.286263 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.29.145.3:46845)
0811 11:10:35.284611 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.284616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.284626 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:35.284652 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.284655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.284665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.284756 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.285885 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.285894 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.285894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.285899 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.285936 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.285965 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.286055 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.286143 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.286160 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260811 11:10:35.286278 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:46845 (local address 127.0.0.1:43948)
0811 11:10:35.284221 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.284268 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.284352 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.285104 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.285112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.285119 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.285700 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.285700 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.285733 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.285759 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.286047 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.286054 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.286057 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.286129 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.286150 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260811 11:10:35.286407 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.29.145.2:44287)
0811 11:10:35.284554 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.284559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.284848 (+ 289us) server_negotiation.cc:407] Connection header received
0811 11:10:35.284879 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.284881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.284892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.284956 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.286031 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.286039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.286040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.286045 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.286078 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.286106 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.286172 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.286295 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.286311 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260811 11:10:35.286507 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.29.145.1:46225)
0811 11:10:35.284608 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.284611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.284622 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:35.284697 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.284698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.284706 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.284812 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.285889 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.285899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.285899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.285904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.285945 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.285969 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.286028 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.286393 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.286411 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":108,"threads_started":1}
I20260811 11:10:35.286569 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:44287 (local address 127.0.0.1:57874)
0811 11:10:35.284739 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.284748 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.284785 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.285219 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.285220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.285223 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.285929 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.285930 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.285935 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.285944 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.286200 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.286201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.286202 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.286248 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.286262 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260811 11:10:35.286250 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.284095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46225 (local address 127.0.0.1:58218)
0811 11:10:35.284567 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.284579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.284630 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.285134 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.285135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.285137 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.285764 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.285765 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.285770 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.285780 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.286053 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.286056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.286057 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.286130 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.286150 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":107,"threads_started":1}
I20260811 11:10:35.287324 3297 tablet_service.cc:1467] Processing CreateTablet for tablet ca4097c6b71348bf8bb4759c082e1321 (DEFAULT_TABLE table=test-table [id=b595a957636b4b9c9e1ec459b96352bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:35.287568 3141 tablet_service.cc:1467] Processing CreateTablet for tablet ca4097c6b71348bf8bb4759c082e1321 (DEFAULT_TABLE table=test-table [id=b595a957636b4b9c9e1ec459b96352bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:35.287707 3297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4097c6b71348bf8bb4759c082e1321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.287920 3141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4097c6b71348bf8bb4759c082e1321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.290545 3538 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Bootstrap starting.
I20260811 11:10:35.290591 3539 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: Bootstrap starting.
I20260811 11:10:35.291265 3539 tablet_bootstrap.cc:654] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.291340 3538 tablet_bootstrap.cc:654] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.291230 3445 tablet_service.cc:1467] Processing CreateTablet for tablet ca4097c6b71348bf8bb4759c082e1321 (DEFAULT_TABLE table=test-table [id=b595a957636b4b9c9e1ec459b96352bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:35.291605 3445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4097c6b71348bf8bb4759c082e1321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.291610 3539 log.cc:824] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.291651 3538 log.cc:824] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.292474 3538 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: No bootstrap required, opened a new log
I20260811 11:10:35.292487 3539 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: No bootstrap required, opened a new log
I20260811 11:10:35.292547 3538 ts_tablet_manager.cc:1397] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:35.292567 3539 ts_tablet_manager.cc:1397] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:35.294664 3539 raft_consensus.cc:348] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.294828 3539 raft_consensus.cc:374] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.294874 3539 raft_consensus.cc:729] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15eaeed1475042469de5af384004d4d7, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.295011 3539 consensus_queue.cc:260] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.295292 3539 ts_tablet_manager.cc:1428] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:35.295274 3542 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Bootstrap starting.
I20260811 11:10:35.295231 3538 raft_consensus.cc:348] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.295419 3222 heartbeater.cc:503] Master 127.29.145.62:44691 was elected leader, sending a full tablet report...
I20260811 11:10:35.295418 3538 raft_consensus.cc:374] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.295460 3538 raft_consensus.cc:729] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a49c9c3cc9664dbf904d7c0240bda07c, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.295578 3538 consensus_queue.cc:260] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.295838 3538 ts_tablet_manager.cc:1428] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 11:10:35.295969 3370 heartbeater.cc:503] Master 127.29.145.62:44691 was elected leader, sending a full tablet report...
I20260811 11:10:35.296193 3542 tablet_bootstrap.cc:654] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.296528 3542 log.cc:824] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.297297 3542 tablet_bootstrap.cc:492] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: No bootstrap required, opened a new log
I20260811 11:10:35.297384 3542 ts_tablet_manager.cc:1397] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:35.299260 3542 raft_consensus.cc:348] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.299407 3542 raft_consensus.cc:374] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.299453 3542 raft_consensus.cc:729] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff8c3edd7dd406eae6f50c1170ea792, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.299594 3542 consensus_queue.cc:260] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.299877 3542 ts_tablet_manager.cc:1428] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:10:35.299901 3522 heartbeater.cc:503] Master 127.29.145.62:44691 was elected leader, sending a full tablet report...
I20260811 11:10:35.302160 3547 raft_consensus.cc:482] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:35.302234 3547 raft_consensus.cc:504] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.302517 3547 leader_election.cc:288] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15eaeed1475042469de5af384004d4d7 (127.29.145.1:46225), a49c9c3cc9664dbf904d7c0240bda07c (127.29.145.2:44287)
I20260811 11:10:35.303952 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.302689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46225 (local address 127.29.145.3:42393)
0811 11:10:35.302744 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.302760 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.302788 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.303042 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.303043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.303045 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.303553 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.303554 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.303563 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.303577 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.303799 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.303800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.303803 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.303862 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.303876 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:10:35.303982 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.302737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:42393 (local address 127.29.145.1:46225)
0811 11:10:35.302941 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.302943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.302952 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:35.302977 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.302978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.302983 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.303035 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.303674 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.303675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.303676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.303681 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.303701 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.303707 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.303776 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.303874 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.303889 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260811 11:10:35.304061 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.302948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:47313 (local address 127.29.145.2:44287)
0811 11:10:35.303009 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.303010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.303020 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:35.303043 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.303044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.303049 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.303109 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.303833 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.303835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.303835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.303840 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.303859 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.303864 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.303920 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.303971 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.303985 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:10:35.304302 3164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4097c6b71348bf8bb4759c082e1321" candidate_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaeed1475042469de5af384004d4d7" is_pre_election: true
I20260811 11:10:35.304368 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.302727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:44287 (local address 127.29.145.3:47313)
0811 11:10:35.302864 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.302876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.302927 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.303129 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.303130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.303132 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:35.303730 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:10:35.303731 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:35.303738 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:10:35.303750 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:35.303923 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:35.303924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:35.303924 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 11:10:35.303972 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 11:10:35.303985 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260811 11:10:35.304416 3164 raft_consensus.cc:2454] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff8c3edd7dd406eae6f50c1170ea792 in term 0.
I20260811 11:10:35.304682 3390 leader_election.cc:302] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15eaeed1475042469de5af384004d4d7, 3ff8c3edd7dd406eae6f50c1170ea792; no voters:
I20260811 11:10:35.304751 3318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4097c6b71348bf8bb4759c082e1321" candidate_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" is_pre_election: true
I20260811 11:10:35.304824 3547 raft_consensus.cc:2781] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:35.304876 3547 raft_consensus.cc:482] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:35.304886 3318 raft_consensus.cc:2454] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff8c3edd7dd406eae6f50c1170ea792 in term 0.
I20260811 11:10:35.304903 3547 raft_consensus.cc:3037] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.305905 3547 raft_consensus.cc:504] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.306056 3547 leader_election.cc:288] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [CANDIDATE]: Term 1 election: Requested vote from peers 15eaeed1475042469de5af384004d4d7 (127.29.145.1:46225), a49c9c3cc9664dbf904d7c0240bda07c (127.29.145.2:44287)
I20260811 11:10:35.306205 3164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4097c6b71348bf8bb4759c082e1321" candidate_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15eaeed1475042469de5af384004d4d7"
I20260811 11:10:35.306227 3318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4097c6b71348bf8bb4759c082e1321" candidate_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a49c9c3cc9664dbf904d7c0240bda07c"
I20260811 11:10:35.306286 3164 raft_consensus.cc:3037] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.306298 3318 raft_consensus.cc:3037] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.307189 3318 raft_consensus.cc:2454] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff8c3edd7dd406eae6f50c1170ea792 in term 1.
I20260811 11:10:35.307186 3164 raft_consensus.cc:2454] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff8c3edd7dd406eae6f50c1170ea792 in term 1.
I20260811 11:10:35.307339 3390 leader_election.cc:302] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15eaeed1475042469de5af384004d4d7, 3ff8c3edd7dd406eae6f50c1170ea792; no voters:
I20260811 11:10:35.307435 3547 raft_consensus.cc:2781] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:35.307587 3547 raft_consensus.cc:686] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 1 LEADER]: Becoming Leader. State: Replica: 3ff8c3edd7dd406eae6f50c1170ea792, State: Running, Role: LEADER
I20260811 11:10:35.307658 3547 consensus_queue.cc:237] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } }
I20260811 11:10:35.308470 3001 catalog_manager.cc:5495] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ff8c3edd7dd406eae6f50c1170ea792 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff8c3edd7dd406eae6f50c1170ea792" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:35.315084 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.314120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.29.145.3:46845)
0811 11:10:35.314170 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.314172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.314193 (+ 21us) server_negotiation.cc:407] Connection header received
0811 11:10:35.314227 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.314228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.314241 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.314328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.314867 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.314868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.314869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.314873 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.314887 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.314891 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.314943 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.314994 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.315008 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:10:35.317081 3318 raft_consensus.cc:1261] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 1 FOLLOWER]: Refusing update from remote peer 3ff8c3edd7dd406eae6f50c1170ea792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:35.317085 3164 raft_consensus.cc:1261] T ca4097c6b71348bf8bb4759c082e1321 P 15eaeed1475042469de5af384004d4d7 [term 1 FOLLOWER]: Refusing update from remote peer 3ff8c3edd7dd406eae6f50c1170ea792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:35.317463 3549 consensus_queue.cc:1035] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a49c9c3cc9664dbf904d7c0240bda07c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:35.317628 3547 consensus_queue.cc:1035] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15eaeed1475042469de5af384004d4d7" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:35.322486 3000 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:35.322592 3000 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:35.323397 3000 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b93d1daba154d86968618d7a5cb3f4d: Sending DeleteTablet for 3 replicas of tablet ca4097c6b71348bf8bb4759c082e1321
I20260811 11:10:35.323740 3141 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4097c6b71348bf8bb4759c082e1321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:35 UTC) from {username='slave'} at 127.0.0.1:58218
I20260811 11:10:35.323803 3297 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4097c6b71348bf8bb4759c082e1321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:35 UTC) from {username='slave'} at 127.0.0.1:57874
I20260811 11:10:35.324013 3445 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4097c6b71348bf8bb4759c082e1321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:35 UTC) from {username='slave'} at 127.0.0.1:43948
I20260811 11:10:35.324069 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3066
I20260811 11:10:35.324278 3564 tablet_replica.cc:330] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: stopping tablet replica
I20260811 11:10:35.324378 3564 raft_consensus.cc:2229] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:35.324466 3564 raft_consensus.cc:2258] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:35.326792 3562 tablet_replica.cc:330] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: stopping tablet replica
I20260811 11:10:35.326929 3562 raft_consensus.cc:2229] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:35.326988 3564 ts_tablet_manager.cc:1905] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:35.327071 3562 raft_consensus.cc:2258] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:35.328223 3562 ts_tablet_manager.cc:1905] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:35.328856 3564 ts_tablet_manager.cc:1918] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:35.328925 3564 log.cc:1196] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/ca4097c6b71348bf8bb4759c082e1321
I20260811 11:10:35.329223 3564 ts_tablet_manager.cc:1939] T ca4097c6b71348bf8bb4759c082e1321 P 3ff8c3edd7dd406eae6f50c1170ea792: Deleting consensus metadata
I20260811 11:10:35.329670 2985 catalog_manager.cc:4841] TS 3ff8c3edd7dd406eae6f50c1170ea792 (127.29.145.3:46845): tablet ca4097c6b71348bf8bb4759c082e1321 (table test-table [id=b595a957636b4b9c9e1ec459b96352bd]) successfully deleted
I20260811 11:10:35.330344 3562 ts_tablet_manager.cc:1918] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:35.330415 3562 log.cc:1196] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/ca4097c6b71348bf8bb4759c082e1321
I20260811 11:10:35.330672 3562 ts_tablet_manager.cc:1939] T ca4097c6b71348bf8bb4759c082e1321 P a49c9c3cc9664dbf904d7c0240bda07c: Deleting consensus metadata
I20260811 11:10:35.331180 2983 catalog_manager.cc:4841] TS a49c9c3cc9664dbf904d7c0240bda07c (127.29.145.2:44287): tablet ca4097c6b71348bf8bb4759c082e1321 (table test-table [id=b595a957636b4b9c9e1ec459b96352bd]) successfully deleted
W20260811 11:10:35.335991 2984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:46225 (error 108)
I20260811 11:10:35.336215 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3225
I20260811 11:10:35.338230 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.338060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46225 (local address 127.0.0.1:58230)
0811 11:10:35.338137 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.338189 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:10:35.338353 2984 catalog_manager.cc:4584] TS 15eaeed1475042469de5af384004d4d7 (127.29.145.1:46225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca4097c6b71348bf8bb4759c082e1321: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46225: connect: Connection refused (error 111)
I20260811 11:10:35.344070 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3374
I20260811 11:10:35.352391 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 2965
2026-08-11T11:10:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (651 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 11:10:35.374971 30276 mini_oidc.cc:152] Starting JWKS server
I20260811 11:10:35.376188 30276 webserver.cc:460] Webserver started at https://127.0.0.1:46077/ using document root <none> and password file <none>
I20260811 11:10:35.376298 30276 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:10:35.376499 30276 webserver.cc:460] Webserver started at http://127.0.0.1:33517/ using document root <none> and password file <none>
2026-08-11T11:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:35Z Disabled control of system clock
I20260811 11:10:35.385744 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37363
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:37645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37363
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46077/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/testchainca.cert with env {}
W20260811 11:10:35.467694 3573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:35.467943 3573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:35.467981 3573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:35.469131 3573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:35.469198 3573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:35.469221 3573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:35.469247 3573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:35.469280 3573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:10:35.469302 3573 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:10:35.469327 3573 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46077/jwks.json
I20260811 11:10:35.470911 3573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46077/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:35.471184 3573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:35.471410 3573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:35.476374 3582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.476374 3578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.476415 3579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.476765 3573 server_base.cc:1034] running on GCE node
I20260811 11:10:35.477026 3573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:35.477291 3573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:35.478437 3573 hybrid_clock.cc:648] HybridClock initialized: now 1786446635478425 us; error 27 us; skew 500 ppm
I20260811 11:10:35.479758 3573 webserver.cc:460] Webserver started at http://127.29.145.62:33005/ using document root <none> and password file <none>
I20260811 11:10:35.480018 3573 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:35.480064 3573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:35.480207 3573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:35.481273 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "6b768f2dcfb74d34b07894b71231b0e0"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "55136f0e7d7c70884cbdaee93fd63aab"
server_key_iv: "dbb017af4aec4d482ee7a79f358c5b21"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.481657 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "6b768f2dcfb74d34b07894b71231b0e0"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "55136f0e7d7c70884cbdaee93fd63aab"
server_key_iv: "dbb017af4aec4d482ee7a79f358c5b21"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.483142 3573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:35.483906 3589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.484097 3573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:35.484203 3573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "6b768f2dcfb74d34b07894b71231b0e0"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "55136f0e7d7c70884cbdaee93fd63aab"
server_key_iv: "dbb017af4aec4d482ee7a79f358c5b21"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.484294 3573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.502231 3573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.502568 3573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.502728 3573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.506676 3573 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37363
I20260811 11:10:35.506717 3655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37363 every 8 connection(s)
I20260811 11:10:35.507076 3573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:35.508046 3656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.510499 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3573
I20260811 11:10:35.510540 3656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: Bootstrap starting.
I20260811 11:10:35.510605 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:35.510816 30276 external_mini_cluster.cc:1428] Setting key 7f39452457565aa2669784c315fc1081
I20260811 11:10:35.511332 3656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.511663 3656 log.cc:824] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.512410 3656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: No bootstrap required, opened a new log
I20260811 11:10:35.514503 3656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } }
I20260811 11:10:35.514683 3656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.514711 3656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b768f2dcfb74d34b07894b71231b0e0, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.514856 3656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } }
I20260811 11:10:35.514930 3656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:35.514951 3656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:35.515017 3656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.515810 3656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } }
I20260811 11:10:35.515929 3656 leader_election.cc:302] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b768f2dcfb74d34b07894b71231b0e0; no voters:
I20260811 11:10:35.516108 3656 leader_election.cc:288] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:35.516170 3662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:35.516347 3662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [term 1 LEADER]: Becoming Leader. State: Replica: 6b768f2dcfb74d34b07894b71231b0e0, State: Running, Role: LEADER
I20260811 11:10:35.516445 3656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:35.516857 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.511347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.29.145.62:37363)
0811 11:10:35.511566 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.511573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.511597 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:10:35.512062 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.512069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.512128 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.512264 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.512531 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.513421 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.514585 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.515213 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.515300 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.516007 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.516011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.516011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.516229 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.516273 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.516288 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.516356 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.516561 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.516638 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260811 11:10:35.516448 3662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } }
I20260811 11:10:35.517938 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:10:35.518435 3664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b768f2dcfb74d34b07894b71231b0e0. Latest consensus state: current_term: 1 leader_uuid: "6b768f2dcfb74d34b07894b71231b0e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } } }
I20260811 11:10:35.518474 3663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b768f2dcfb74d34b07894b71231b0e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b768f2dcfb74d34b07894b71231b0e0" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37363 } } }
I20260811 11:10:35.518985 3664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:35.518989 3663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0 [sys.catalog]: This master's current role is: LEADER
W20260811 11:10:35.519200 3677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:10:35.519254 3677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:10:35.519330 3680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:35.519515 3680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:35.521155 3680 catalog_manager.cc:1347] Generated new cluster ID: f1ca1231df474906bab97ed4e8cfbff4
I20260811 11:10:35.521238 3680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:35.528004 3680 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:35.528514 3680 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:35.535661 3680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b768f2dcfb74d34b07894b71231b0e0: Generated new TSK 0
I20260811 11:10:35.535862 3680 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 11:10:35.571805 3690 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 11:10:35.571942 3690 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 11:10:35.572240 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.554321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.29.145.62:37363)
0811 11:10:35.554393 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.554396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.554549 (+ 153us) server_negotiation.cc:407] Connection header received
0811 11:10:35.554634 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.554635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.554650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.554755 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:10:35.554981 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.555803 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.556709 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.557044 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.557127 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.557427 (+ 300us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:10:35.571965 (+ 14538us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 11:10:35.572147 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 11:10:35.572329 3659 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38316: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 11:10:35.572707 3689 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:35.554182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37363 (local address 127.0.0.1:38316)
0811 11:10:35.554470 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.554480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.554548 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.554828 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.554830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.554832 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:10:35.554903 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:35.554905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.555921 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.555923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:35.556604 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:35.556606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.557188 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.557189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:35.557257 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.557287 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:10:35.572159 (+ 14872us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 11:10:35.572273 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.145.62:37363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":160,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.29.145.62:37363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46077/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 11:10:35.573282 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3573
2026-08-11T11:10:35Z chronyd exiting
I20260811 11:10:35.613332 30276 test_util.cc:175] -----------------------------------------------
I20260811 11:10:35.613420 30276 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786446611237378-30276-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 11:10:35.614310 30276 mini_oidc.cc:152] Starting JWKS server
I20260811 11:10:35.614979 30276 webserver.cc:460] Webserver started at https://127.0.0.1:43245/ using document root <none> and password file <none>
I20260811 11:10:35.615015 30276 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:10:35.615181 30276 webserver.cc:460] Webserver started at http://127.0.0.1:40755/ using document root <none> and password file <none>
2026-08-11T11:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:35Z Disabled control of system clock
I20260811 11:10:35.624995 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:33909
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:33909
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43245/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/testchainca.cert with env {}
W20260811 11:10:35.709852 3696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:35.710057 3696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:35.710078 3696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:35.711150 3696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:35.711184 3696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:35.711196 3696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:35.711208 3696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:35.711220 3696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:10:35.711233 3696 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:10:35.711244 3696 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43245/jwks.json
I20260811 11:10:35.712692 3696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:33909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:33909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43245/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:35.712904 3696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:35.713070 3696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:35.718086 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.718065 3702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.718065 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.718302 3696 server_base.cc:1034] running on GCE node
I20260811 11:10:35.718545 3696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:35.718837 3696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:35.719981 3696 hybrid_clock.cc:648] HybridClock initialized: now 1786446635719970 us; error 38 us; skew 500 ppm
I20260811 11:10:35.721187 3696 webserver.cc:460] Webserver started at http://127.29.145.62:44747/ using document root <none> and password file <none>
I20260811 11:10:35.721473 3696 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:35.721521 3696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:35.721655 3696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:35.722765 3696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "72919017dd50464eba69141398b9836b"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "be3d7a27d04da356f8873173b640453f"
server_key_iv: "0468dba7a80beb883c5d43dfb1f8386f"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.723135 3696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "72919017dd50464eba69141398b9836b"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "be3d7a27d04da356f8873173b640453f"
server_key_iv: "0468dba7a80beb883c5d43dfb1f8386f"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.724488 3696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:35.725343 3710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.725572 3696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:35.725651 3696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "72919017dd50464eba69141398b9836b"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "be3d7a27d04da356f8873173b640453f"
server_key_iv: "0468dba7a80beb883c5d43dfb1f8386f"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.725747 3696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.739717 3696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.740233 3696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.740397 3696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.744941 3696 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:33909
I20260811 11:10:35.744975 3783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:33909 every 8 connection(s)
I20260811 11:10:35.745343 3696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:35.746160 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.748833 3784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b: Bootstrap starting.
I20260811 11:10:35.749383 3784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.749740 3784 log.cc:824] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.750584 3784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b: No bootstrap required, opened a new log
I20260811 11:10:35.751223 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3696
I20260811 11:10:35.751839 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:35.752164 30276 external_mini_cluster.cc:1428] Setting key 9417500dfa67897cd2ad1b599c6a6f15
I20260811 11:10:35.752548 3784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } }
I20260811 11:10:35.752753 3784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.752806 3784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72919017dd50464eba69141398b9836b, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.752944 3784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } }
I20260811 11:10:35.753036 3784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:35.753060 3784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:35.753132 3784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.753968 3784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } }
I20260811 11:10:35.754062 3784 leader_election.cc:302] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72919017dd50464eba69141398b9836b; no voters:
I20260811 11:10:35.754266 3784 leader_election.cc:288] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:35.754488 3790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:35.754559 3784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:35.754608 3790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [term 1 LEADER]: Becoming Leader. State: Replica: 72919017dd50464eba69141398b9836b, State: Running, Role: LEADER
I20260811 11:10:35.754698 3790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } }
I20260811 11:10:35.755056 3790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72919017dd50464eba69141398b9836b. Latest consensus state: current_term: 1 leader_uuid: "72919017dd50464eba69141398b9836b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } } }
I20260811 11:10:35.755152 3790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:35.755332 3791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72919017dd50464eba69141398b9836b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72919017dd50464eba69141398b9836b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33909 } } }
I20260811 11:10:35.755412 3791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:35.755525 3800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:35.756139 3800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:35.758177 3800 catalog_manager.cc:1347] Generated new cluster ID: 48e9c91b6a824a4e8f9dc77baaa4f62f
I20260811 11:10:35.758248 3800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:35.758630 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.752804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.29.145.62:33909)
0811 11:10:35.753049 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.753059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.753095 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:10:35.753290 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.753296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.753349 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.753462 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.753745 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.755175 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.756346 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.756927 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.757056 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.757863 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.757867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.757867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.758095 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.758138 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.758157 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.758234 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.758334 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.758416 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260811 11:10:35.759624 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:10:35.763170 3800 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:35.763676 3800 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:35.771835 3800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72919017dd50464eba69141398b9836b: Generated new TSK 0
I20260811 11:10:35.772064 3800 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 11:10:35.797430 3817 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 11:10:35.797533 3817 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 11:10:35.797952 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.783146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.29.145.62:33909)
0811 11:10:35.783197 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.783199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.783296 (+ 97us) server_negotiation.cc:407] Connection header received
0811 11:10:35.783362 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.783363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.783376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.783453 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:10:35.783594 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.784403 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.785212 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.785564 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.785638 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.785851 (+ 213us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:10:35.797688 (+ 11837us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43245/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 11:10:35.797875 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43245/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 11:10:35.798017 3789 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43245/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 11:10:35.798059 3816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:35.783057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:33909 (local address 127.0.0.1:56494)
0811 11:10:35.783207 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:35.783217 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:35.783281 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:35.783457 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:35.783458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:35.783460 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:10:35.783518 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:35.783520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.784514 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.784516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:35.785110 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:35.785112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.785648 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.785649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:35.785692 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.785697 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:10:35.797849 (+ 12152us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43245/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 11:10:35.797958 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.145.62:33909: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43245/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
I20260811 11:10:35.798429 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3696
2026-08-11T11:10:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 11:10:35.839071 30276 mini_oidc.cc:152] Starting JWKS server
I20260811 11:10:35.839808 30276 webserver.cc:460] Webserver started at https://127.0.0.1:33671/ using document root <none> and password file <none>
I20260811 11:10:35.839850 30276 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:10:35.840020 30276 webserver.cc:460] Webserver started at http://127.0.0.1:34081/ using document root <none> and password file <none>
2026-08-11T11:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:35Z Disabled control of system clock
I20260811 11:10:35.850385 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:41721
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:41721
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 11:10:35.938395 3823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:35.938601 3823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:35.938622 3823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:35.939781 3823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:35.939826 3823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:35.939838 3823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:35.939850 3823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:35.939863 3823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:10:35.939875 3823 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:10:35.939930 3823 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33671/jwks.json
I20260811 11:10:35.941277 3823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:41721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:41721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:35.941532 3823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:35.941875 3823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:35.946834 3828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.946897 3833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:35.946844 3829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:35.947319 3823 server_base.cc:1034] running on GCE node
I20260811 11:10:35.947594 3823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:35.947923 3823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:35.949100 3823 hybrid_clock.cc:648] HybridClock initialized: now 1786446635949091 us; error 39 us; skew 500 ppm
I20260811 11:10:35.950529 3823 webserver.cc:460] Webserver started at http://127.29.145.62:40029/ using document root <none> and password file <none>
I20260811 11:10:35.950791 3823 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:35.950867 3823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:35.951025 3823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:35.952149 3823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "aa7d324eab1949888c763e465faef437"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "d214322e7a4eadaca2325ef704c4b32d"
server_key_iv: "cae7af0bd58419060156988f8c9c24c3"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.952526 3823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "aa7d324eab1949888c763e465faef437"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "d214322e7a4eadaca2325ef704c4b32d"
server_key_iv: "cae7af0bd58419060156988f8c9c24c3"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.953933 3823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:35.954831 3839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:35.955036 3823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:35.955152 3823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "aa7d324eab1949888c763e465faef437"
format_stamp: "Formatted at 2026-08-11 11:10:35 on dist-test-slave-69rg"
server_key: "d214322e7a4eadaca2325ef704c4b32d"
server_key_iv: "cae7af0bd58419060156988f8c9c24c3"
server_key_version: "encryptionkey@0"
I20260811 11:10:35.955252 3823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:35.971223 3823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:35.971590 3823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:35.971756 3823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:35.976066 3823 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:41721
I20260811 11:10:35.976106 3900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:41721 every 8 connection(s)
I20260811 11:10:35.976483 3823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:35.977178 3901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:35.979898 3901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437: Bootstrap starting.
I20260811 11:10:35.980435 3901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:35.980684 3901 log.cc:824] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:35.981355 3901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437: No bootstrap required, opened a new log
I20260811 11:10:35.982980 3901 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } }
I20260811 11:10:35.983120 3901 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:35.983142 3901 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7d324eab1949888c763e465faef437, State: Initialized, Role: FOLLOWER
I20260811 11:10:35.983284 3901 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } }
I20260811 11:10:35.983369 3901 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:35.983393 3901 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:35.983423 3901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:35.983932 3901 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } }
I20260811 11:10:35.984016 3901 leader_election.cc:302] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa7d324eab1949888c763e465faef437; no voters:
I20260811 11:10:35.984138 3901 leader_election.cc:288] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:35.984218 3908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:35.984411 3901 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:35.984395 3908 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [term 1 LEADER]: Becoming Leader. State: Replica: aa7d324eab1949888c763e465faef437, State: Running, Role: LEADER
I20260811 11:10:35.984606 3908 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } }
I20260811 11:10:35.984962 3909 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa7d324eab1949888c763e465faef437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } } }
I20260811 11:10:35.985119 3910 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa7d324eab1949888c763e465faef437. Latest consensus state: current_term: 1 leader_uuid: "aa7d324eab1949888c763e465faef437" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa7d324eab1949888c763e465faef437" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 41721 } } }
I20260811 11:10:35.985198 3910 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:35.985113 3909 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:35.985595 3917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:35.985953 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3823
I20260811 11:10:35.986063 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:35.986119 3917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:35.986267 30276 external_mini_cluster.cc:1428] Setting key f83e180450648786881874dd2eee9907
I20260811 11:10:35.987869 3917 catalog_manager.cc:1347] Generated new cluster ID: df915425c91649d9b1d1698ca1889fcf
I20260811 11:10:35.987926 3917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:35.991524 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:35.986901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43650 (local address 127.29.145.62:41721)
0811 11:10:35.987080 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:35.987088 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:35.987116 (+ 28us) server_negotiation.cc:407] Connection header received
0811 11:10:35.987256 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:35.987264 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:35.987346 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:35.987496 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:35.987756 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.988742 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.989522 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:35.990009 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:35.990140 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:35.990851 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:35.990855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:35.990855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:35.991034 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:35.991065 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:35.991075 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:35.991153 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:35.991235 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 11:10:35.991317 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260811 11:10:35.992467 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:10:35.999989 3917 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:36.000695 3917 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:36.012879 3917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa7d324eab1949888c763e465faef437: Generated new TSK 0
I20260811 11:10:36.013110 3917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:10:36.024345 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:36.020814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.29.145.62:41721)
0811 11:10:36.020886 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:36.020888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:36.021015 (+ 127us) server_negotiation.cc:407] Connection header received
0811 11:10:36.021102 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:36.021103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:36.021117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:36.021210 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:10:36.021339 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.021957 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.022753 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.022990 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.023072 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:36.023270 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:10:36.024104 (+ 834us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 11:10:36.024271 (+ 167us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 11:10:36.024422 3928 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 11:10:36.024535 3936 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:36.020738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:41721 (local address 127.0.0.1:43664)
0811 11:10:36.020940 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:36.020951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:36.021034 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:36.021211 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:36.021212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:36.021214 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:10:36.021270 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:36.021272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.022047 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.022048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:36.022656 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:36.022658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.023082 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.023084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:36.023142 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:36.023150 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:10:36.024214 (+ 1064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 11:10:36.024311 (+ 97us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.145.62:41721: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260811 11:10:36.024794 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3823
E20260811 11:10:36.040645 3939 webserver.cc:533] Webserver: sslize failed:
E20260811 11:10:36.040740 3939 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T11:10:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (230 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T11:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:36Z Disabled control of system clock
I20260811 11:10:36.080832 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34083
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:34083
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/testchainca.cert with env {}
W20260811 11:10:36.171459 3943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:36.171738 3943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:36.171772 3943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:36.173117 3943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:36.173171 3943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:36.173185 3943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:36.173199 3943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:36.173218 3943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:10:36.173230 3943 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:10:36.173242 3943 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 11:10:36.175035 3943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:34083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:36.175325 3943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:36.175566 3943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:36.181574 3952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:36.181583 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:36.181584 3950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:36.182255 3943 server_base.cc:1034] running on GCE node
I20260811 11:10:36.182806 3943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:36.183171 3943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:36.184633 3943 hybrid_clock.cc:648] HybridClock initialized: now 1786446636184436 us; error 215 us; skew 500 ppm
I20260811 11:10:36.186411 3943 webserver.cc:460] Webserver started at http://127.29.145.62:35527/ using document root <none> and password file <none>
I20260811 11:10:36.186733 3943 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:36.186825 3943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:36.186976 3943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:36.188243 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "50e0216eb8d947bdb849cd7ff2e916e3"
format_stamp: "Formatted at 2026-08-11 11:10:36 on dist-test-slave-69rg"
server_key: "0596e193ab4d1cc46a84e03f28083307"
server_key_iv: "daf37d16f9494f7edc7f56528a0c1578"
server_key_version: "encryptionkey@0"
I20260811 11:10:36.188659 3943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "50e0216eb8d947bdb849cd7ff2e916e3"
format_stamp: "Formatted at 2026-08-11 11:10:36 on dist-test-slave-69rg"
server_key: "0596e193ab4d1cc46a84e03f28083307"
server_key_iv: "daf37d16f9494f7edc7f56528a0c1578"
server_key_version: "encryptionkey@0"
I20260811 11:10:36.190465 3943 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 11:10:36.191613 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:36.191936 3943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:36.192070 3943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "50e0216eb8d947bdb849cd7ff2e916e3"
format_stamp: "Formatted at 2026-08-11 11:10:36 on dist-test-slave-69rg"
server_key: "0596e193ab4d1cc46a84e03f28083307"
server_key_iv: "daf37d16f9494f7edc7f56528a0c1578"
server_key_version: "encryptionkey@0"
I20260811 11:10:36.192159 3943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:36.226002 3943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:36.226465 3943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:36.226787 3943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:36.231386 3943 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:34083
I20260811 11:10:36.231418 4020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:34083 every 8 connection(s)
I20260811 11:10:36.231779 3943 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:36.232985 4021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:36.235713 4021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3: Bootstrap starting.
I20260811 11:10:36.236373 4021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:36.236654 4021 log.cc:824] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:36.236842 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 3943
I20260811 11:10:36.236960 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:36.237157 30276 external_mini_cluster.cc:1428] Setting key 2fbccbb9816736ee40aeca150222192d
I20260811 11:10:36.237607 4021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3: No bootstrap required, opened a new log
I20260811 11:10:36.240546 4021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } }
I20260811 11:10:36.240798 4021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:36.240844 4021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e0216eb8d947bdb849cd7ff2e916e3, State: Initialized, Role: FOLLOWER
I20260811 11:10:36.240979 4021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } }
I20260811 11:10:36.241084 4021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:36.241127 4021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:36.241179 4021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:36.243798 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:36.237763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.29.145.62:34083)
0811 11:10:36.238000 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:36.238007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:36.238032 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:10:36.239101 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:36.239110 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:36.239173 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:36.239321 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:36.239625 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.240510 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.241737 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.242295 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.242404 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:36.243080 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:36.243084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:36.243084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:10:36.243306 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:36.243341 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:10:36.243352 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:36.243406 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:36.243481 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 11:10:36.243580 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
I20260811 11:10:36.244848 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:10:36.277308 4021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } }
I20260811 11:10:36.277580 4021 leader_election.cc:302] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50e0216eb8d947bdb849cd7ff2e916e3; no voters:
I20260811 11:10:36.278002 4021 leader_election.cc:288] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:36.278298 4028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:36.278288 4029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } } }
I20260811 11:10:36.278393 4021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:36.278475 4029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:36.278543 4028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [term 1 LEADER]: Becoming Leader. State: Replica: 50e0216eb8d947bdb849cd7ff2e916e3, State: Running, Role: LEADER
I20260811 11:10:36.278650 4028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } }
I20260811 11:10:36.279050 4029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50e0216eb8d947bdb849cd7ff2e916e3. Latest consensus state: current_term: 1 leader_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50e0216eb8d947bdb849cd7ff2e916e3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34083 } } }
I20260811 11:10:36.279134 4029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:36.279657 4037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:36.280251 4037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:36.282147 4037 catalog_manager.cc:1347] Generated new cluster ID: a30658d966d7436eb574406a8aa100c0
I20260811 11:10:36.282209 4037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:36.290572 4037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:36.291162 4037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:36.308830 4037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50e0216eb8d947bdb849cd7ff2e916e3: Generated new TSK 0
I20260811 11:10:36.309063 4037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:10:36.370313 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:36.316520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.29.145.62:34083)
0811 11:10:36.316585 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:36.316587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:36.316721 (+ 134us) server_negotiation.cc:407] Connection header received
0811 11:10:36.316769 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:36.316770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:36.316783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:36.316868 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:10:36.316995 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.317807 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.318704 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.319041 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.319118 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:36.319360 (+ 242us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:10:36.370016 (+ 50656us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 11:10:36.370226 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47902: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 11:10:36.370411 4027 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47902: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 11:10:36.370553 4054 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:36.316355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34083 (local address 127.0.0.1:47902)
0811 11:10:36.316652 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:36.316661 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:36.316716 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:36.316876 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:36.316877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:36.316879 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:10:36.316930 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:36.316932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.317939 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.317940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:36.318610 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:36.318613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:36.319189 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:36.319190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:36.319241 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:36.319246 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:10:36.370251 (+ 51005us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 11:10:36.370412 (+ 161us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.29.145.62:34083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":165,"threads_started":1}
I20260811 11:10:36.371042 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 3943
2026-08-11T11:10:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (322 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (95 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (94 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:37 dist-test-slave-69rg krb5kdc[4066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:37 dist-test-slave-69rg krb5kdc[4066](info): set up 2 sockets
Aug 11 11:10:37 dist-test-slave-69rg krb5kdc[4066](info): commencing operation
krb5kdc: starting...
W20260811 11:10:39.087967 30276 mini_kdc.cc:121] Time spent starting KDC: real 2.505s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:39.238394 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42767
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:39.337466 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:39.337795 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:39.337836 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:39.339217 4080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:39.339294 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:39.339320 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:39.339349 4080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:39.339381 4080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:39.341058 4080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:39.341392 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:39.341683 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:39.347281 4086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.347306 4090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.347461 4087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:39.347859 4080 server_base.cc:1034] running on GCE node
I20260811 11:10:39.348165 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:39.348502 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:39.349723 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786446639349704 us; error 69 us; skew 500 ppm
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:39.353425 4080 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:39.354097 4080 webserver.cc:460] Webserver started at http://127.29.145.62:37563/ using document root <none> and password file <none>
I20260811 11:10:39.354393 4080 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:39.354472 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:39.354651 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:39.356057 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "028949404a0e48279d14614c1f46008d"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "14ded585dd7ca1557283b6fad9d3912f"
server_key_iv: "7b63dbf4fc379a939c502ceef5f6dfc5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.356526 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "028949404a0e48279d14614c1f46008d"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "14ded585dd7ca1557283b6fad9d3912f"
server_key_iv: "7b63dbf4fc379a939c502ceef5f6dfc5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.358376 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 11:10:39.359308 4099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.359571 4080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:39.359701 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "028949404a0e48279d14614c1f46008d"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "14ded585dd7ca1557283b6fad9d3912f"
server_key_iv: "7b63dbf4fc379a939c502ceef5f6dfc5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.359797 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:39.369705 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:39.372452 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:39.372676 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:39.378285 4080 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42767
I20260811 11:10:39.378319 4158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42767 every 8 connection(s)
I20260811 11:10:39.378795 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:39.380029 4159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:39.383172 4159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d: Bootstrap starting.
I20260811 11:10:39.384016 4159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:39.384397 4159 log.cc:824] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:39.385346 4159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d: No bootstrap required, opened a new log
I20260811 11:10:39.386072 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4080
I20260811 11:10:39.386176 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:39.386399 30276 external_mini_cluster.cc:1428] Setting key 3ef4ffaff7568b7f58a99cd0f3f9bb05
I20260811 11:10:39.388084 4159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } }
I20260811 11:10:39.388303 4159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:39.388350 4159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 028949404a0e48279d14614c1f46008d, State: Initialized, Role: FOLLOWER
I20260811 11:10:39.388530 4159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } }
I20260811 11:10:39.388614 4159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:39.388639 4159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:39.388713 4159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:39.390091 4159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } }
I20260811 11:10:39.390223 4159 leader_election.cc:302] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 028949404a0e48279d14614c1f46008d; no voters:
I20260811 11:10:39.390409 4159 leader_election.cc:288] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:39.390514 4164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:39.390717 4159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:39.390719 4164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [term 1 LEADER]: Becoming Leader. State: Replica: 028949404a0e48279d14614c1f46008d, State: Running, Role: LEADER
I20260811 11:10:39.390997 4164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } }
I20260811 11:10:39.391481 4165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "028949404a0e48279d14614c1f46008d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } } }
I20260811 11:10:39.391484 4166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 028949404a0e48279d14614c1f46008d. Latest consensus state: current_term: 1 leader_uuid: "028949404a0e48279d14614c1f46008d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028949404a0e48279d14614c1f46008d" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42767 } } }
I20260811 11:10:39.391683 4165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:39.391690 4166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:39.391996 4173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:39.392684 4173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:39.395066 4173 catalog_manager.cc:1347] Generated new cluster ID: 86175f5fb01a41bebbd31237ef9d637a
I20260811 11:10:39.395134 4173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:39.400388 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:39.387150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.29.145.62:42767)
0811 11:10:39.387407 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:39.387422 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:39.387469 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:10:39.388016 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:39.388028 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:39.388106 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:39.388289 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:39.389344 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.390919 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.392229 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.392955 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.393108 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:39.396370 (+ 3262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:39.396375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:39.396376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:39.396389 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:39.398558 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.399007 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.399011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.399011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.399143 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.399373 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.399374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.399374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.399712 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:39.399887 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:39.400026 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 11:10:39.400173 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":124,"threads_started":1}
I20260811 11:10:39.405459 4173 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:39.406060 4173 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:39.413205 4173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 028949404a0e48279d14614c1f46008d: Generated new TSK 0
I20260811 11:10:39.413461 4173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:39.454356 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:39.560161 4189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:39.560421 4189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:39.560448 4189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:39.561631 4189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:39.561667 4189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:39.561695 4189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:39.563369 4189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:39.563656 4189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:39.563881 4189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:39.569722 4195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.569717 4197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.569717 4194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:39.570218 4189 server_base.cc:1034] running on GCE node
I20260811 11:10:39.570514 4189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:39.570850 4189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:39.572031 4189 hybrid_clock.cc:648] HybridClock initialized: now 1786446639572021 us; error 56 us; skew 500 ppm
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:39.575201 4189 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:39.575851 4189 webserver.cc:460] Webserver started at http://127.29.145.1:34285/ using document root <none> and password file <none>
I20260811 11:10:39.576174 4189 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:39.576247 4189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:39.576416 4189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:39.577764 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "33bb44c3581e45bd99ab725024bc2a69"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "bb5615ebfcbaf4eb77b8ddf724153e30"
server_key_iv: "e10daf011ca39f1222927a9c18f7cdd5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.578264 4189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "33bb44c3581e45bd99ab725024bc2a69"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "bb5615ebfcbaf4eb77b8ddf724153e30"
server_key_iv: "e10daf011ca39f1222927a9c18f7cdd5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.579898 4189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:39.580757 4205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.580942 4189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:39.581067 4189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "33bb44c3581e45bd99ab725024bc2a69"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "bb5615ebfcbaf4eb77b8ddf724153e30"
server_key_iv: "e10daf011ca39f1222927a9c18f7cdd5"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.581154 4189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:39.602451 4189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:39.605468 4189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:39.605729 4189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:39.606118 4189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:39.606570 4189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:39.606632 4189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.606678 4189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:39.606709 4189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.614257 4189 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:38003
I20260811 11:10:39.614348 4340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:38003 every 8 connection(s)
I20260811 11:10:39.614811 4189 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:39.623198 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4189
I20260811 11:10:39.623340 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:39.623669 30276 external_mini_cluster.cc:1428] Setting key 917c3fc1d690dec15d92f7dd0e3f141a
I20260811 11:10:39.627305 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:39.615886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:43033 (local address 127.29.145.62:42767)
0811 11:10:39.616007 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:39.616009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:39.616266 (+ 257us) server_negotiation.cc:407] Connection header received
0811 11:10:39.617472 (+ 1206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:39.617474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:39.617495 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:39.617566 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:39.618505 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.619264 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.620352 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.620603 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.620667 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:39.623524 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:39.623526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:39.623527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:39.623540 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:39.625366 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.625994 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.625995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.625996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.626069 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.626394 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.626395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.626395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.626513 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:39.626567 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:39.627019 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 11:10:39.627165 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:10:39.627417 4344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:39.615868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42767 (local address 127.29.145.1:43033)
0811 11:10:39.616107 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:39.616158 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:39.617317 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:39.617675 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:39.617679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:39.618142 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:39.618397 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:39.618402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.619362 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.619363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:39.620260 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:39.620262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.620688 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.620689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:39.620806 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:39.621377 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:39.621387 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:39.623405 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:39.625463 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:39.625474 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:39.625477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:39.625913 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:39.626145 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:39.626146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:39.626147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:39.626319 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:39.626582 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:39.626590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:39.626804 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 11:10:39.626954 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 11:10:39.627202 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":124,"threads_started":1}
I20260811 11:10:39.628113 4341 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42767
I20260811 11:10:39.628249 4341 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:39.628507 4341 heartbeater.cc:511] Master 127.29.145.62:42767 requested a full tablet report, sending...
I20260811 11:10:39.629181 4117 ts_manager.cc:194] Registered new tserver with Master: 33bb44c3581e45bd99ab725024bc2a69 (127.29.145.1:38003)
I20260811 11:10:39.629923 4117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:43033
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:39.657609 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:39.753170 4347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:39.753425 4347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:39.753451 4347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:39.754853 4347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:39.754901 4347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:39.754932 4347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:39.756515 4347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:39.756807 4347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:39.757100 4347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:39.763254 4354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.763254 4355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.763283 4357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:39.763633 4347 server_base.cc:1034] running on GCE node
I20260811 11:10:39.763932 4347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:39.764178 4347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:39.765321 4347 hybrid_clock.cc:648] HybridClock initialized: now 1786446639765310 us; error 31 us; skew 500 ppm
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:39.768407 4347 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:39.769026 4347 webserver.cc:460] Webserver started at http://127.29.145.2:34517/ using document root <none> and password file <none>
I20260811 11:10:39.769323 4347 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:39.769399 4347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:39.769582 4347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:39.770895 4347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "a6614d73cdc44bdcacfdd65d460a8ddb"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "8e43f96b5c8161b66a750660576a0acc"
server_key_iv: "ce5e88d44e47251743919a03fbbc2061"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.771317 4347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "a6614d73cdc44bdcacfdd65d460a8ddb"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "8e43f96b5c8161b66a750660576a0acc"
server_key_iv: "ce5e88d44e47251743919a03fbbc2061"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.772893 4347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:39.773758 4364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.773943 4347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:39.774063 4347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "a6614d73cdc44bdcacfdd65d460a8ddb"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "8e43f96b5c8161b66a750660576a0acc"
server_key_iv: "ce5e88d44e47251743919a03fbbc2061"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.774147 4347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:39.787405 4347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:39.789994 4347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:39.790190 4347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:39.790541 4347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:39.790959 4347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:39.791018 4347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.791069 4347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:39.791096 4347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.798588 4347 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:41889
I20260811 11:10:39.798646 4491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:41889 every 8 connection(s)
I20260811 11:10:39.799043 4347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:39.805328 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4347
I20260811 11:10:39.805473 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:39.805713 30276 external_mini_cluster.cc:1428] Setting key a469d34176ab4b9c405f2c4a7d4020e6
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:39.811053 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:39.799913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:34731 (local address 127.29.145.62:42767)
0811 11:10:39.800017 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:39.800020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:39.800229 (+ 209us) server_negotiation.cc:407] Connection header received
0811 11:10:39.801360 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:39.801362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:39.801378 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:39.801447 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:39.802359 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.803036 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.804123 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.804357 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.804437 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:39.807170 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:39.807172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:39.807172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:39.807182 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:39.809095 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.809711 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.809713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.809714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.809785 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:39.810224 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:39.810225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:39.810226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:39.810373 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:39.810456 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:39.810851 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 11:10:39.810967 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:39.811249 4494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:39.799865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42767 (local address 127.29.145.2:34731)
0811 11:10:39.800092 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:39.800134 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:39.801229 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:39.801538 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:39.801545 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:39.801964 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:39.802259 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:39.802263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.803138 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.803139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:39.804026 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:39.804028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:39.804456 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:39.804457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:39.804611 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:39.805231 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:39.805245 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:39.807068 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:39.809212 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:39.809223 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:39.809226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:39.809609 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:39.809960 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:39.809961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:39.809961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:39.810126 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:39.810461 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:39.810468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:39.810678 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 11:10:39.810819 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 11:10:39.811005 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260811 11:10:39.812016 4492 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42767
I20260811 11:10:39.812140 4492 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:39.812388 4492 heartbeater.cc:511] Master 127.29.145.62:42767 requested a full tablet report, sending...
I20260811 11:10:39.812911 4117 ts_manager.cc:194] Registered new tserver with Master: a6614d73cdc44bdcacfdd65d460a8ddb (127.29.145.2:41889)
I20260811 11:10:39.813544 4117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:34731
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:39.841193 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:39.958132 4498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:39.958508 4498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:39.958556 4498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:39.961035 4498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:39.961107 4498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:39.961201 4498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:39.964346 4498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42767
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:39.964675 4498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:39.965010 4498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:39.971390 4507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.971395 4504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:39.971719 4505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:39.972097 4498 server_base.cc:1034] running on GCE node
I20260811 11:10:39.972478 4498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:39.972882 4498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:39.974094 4498 hybrid_clock.cc:648] HybridClock initialized: now 1786446639974063 us; error 78 us; skew 500 ppm
Aug 11 11:10:39 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:39.976830 4498 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:39.977330 4498 webserver.cc:460] Webserver started at http://127.29.145.3:44937/ using document root <none> and password file <none>
I20260811 11:10:39.977578 4498 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:39.977653 4498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:39.977854 4498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:39.979019 4498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "5ea3b0b355bd4f6b8a572985d2d6e3ec"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "91ff9428ec95b583ebd2ce4028d4b207"
server_key_iv: "8097f5e087f60d061d54a65a178a9114"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.979437 4498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "5ea3b0b355bd4f6b8a572985d2d6e3ec"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "91ff9428ec95b583ebd2ce4028d4b207"
server_key_iv: "8097f5e087f60d061d54a65a178a9114"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.980877 4498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:39.981698 4514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.981909 4498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:39.982007 4498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "5ea3b0b355bd4f6b8a572985d2d6e3ec"
format_stamp: "Formatted at 2026-08-11 11:10:39 on dist-test-slave-69rg"
server_key: "91ff9428ec95b583ebd2ce4028d4b207"
server_key_iv: "8097f5e087f60d061d54a65a178a9114"
server_key_version: "encryptionkey@0"
I20260811 11:10:39.982064 4498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:39.992942 4498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:39.995052 4498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:39.995170 4498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:39.995364 4498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:39.995635 4498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:39.995667 4498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:39.995688 4498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:39.995702 4498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:40.001650 4498 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:43883
I20260811 11:10:40.001754 4645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:43883 every 8 connection(s)
I20260811 11:10:40.002089 4498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:40.010730 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4498
I20260811 11:10:40.010869 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:40.011189 30276 external_mini_cluster.cc:1428] Setting key bbd5be02c6bf9fa9c1f8e46a02fe982d
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446639, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:40.018364 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:40.002880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:57633 (local address 127.29.145.62:42767)
0811 11:10:40.002950 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.002953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.003199 (+ 246us) server_negotiation.cc:407] Connection header received
0811 11:10:40.004140 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.004141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.004154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.004205 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.004857 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.005632 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.006918 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.007145 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.007227 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:40.014951 (+ 7724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:40.014952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:40.014953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:40.014961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:40.016683 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.017240 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.017241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.017242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.017311 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.017568 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.017569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.017569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.017717 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:40.017782 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:40.018199 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 11:10:40.018286 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:10:40.018504 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:40.002891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42767 (local address 127.29.145.3:57633)
0811 11:10:40.003060 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:40.003099 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:40.004035 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:40.004272 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:40.004276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:40.004528 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:40.004734 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:40.004738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.005776 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.005778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:40.006786 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:40.006789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.007295 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.007297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:40.007441 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:40.012895 (+ 5454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:40.012906 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:40.014801 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:40.016803 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:40.016814 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:40.016817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:40.017144 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:40.017390 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:40.017390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:40.017391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:40.017502 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:40.017788 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:40.017794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:40.018006 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 11:10:40.018115 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 11:10:40.018270 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260811 11:10:40.019094 4647 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42767
I20260811 11:10:40.019227 4647 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:40.019446 4647 heartbeater.cc:511] Master 127.29.145.62:42767 requested a full tablet report, sending...
I20260811 11:10:40.019987 4117 ts_manager.cc:194] Registered new tserver with Master: 5ea3b0b355bd4f6b8a572985d2d6e3ec (127.29.145.3:43883)
I20260811 11:10:40.020452 4117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:57633
I20260811 11:10:40.024299 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/test-admin.keytab.
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.044446 30276 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 11:10:40.046494 30276 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc' of type FILE
W20260811 11:10:40.046514 4655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:40.045461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.29.145.1:38003)
0811 11:10:40.045656 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.045663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.045676 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:10:40.045721 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.045723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.045737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.045859 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.046413 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55698: BlockingRecv error: recv got EOF from 127.0.0.1:55698 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":138,"threads_started":1}
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.047575 30276 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.048506 30276 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:40.057178 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:40.049056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.29.145.1:38003)
0811 11:10:40.049122 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.049124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.049167 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:10:40.049236 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.049237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.049250 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.049305 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.049709 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.050535 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.051458 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.051819 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.051909 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:40.054293 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:40.054302 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:40.054303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:40.054310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:40.055873 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.056232 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.056236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.056236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.056297 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.056486 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.056487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.056487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.056635 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:40.056766 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:40.056810 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 11:10:40.056880 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 11:10:40.059209 4655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:40.058617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.29.145.1:38003)
0811 11:10:40.058675 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.058677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.058716 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:10:40.058791 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.058792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.058804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.058853 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.059146 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55718: BlockingRecv error: recv got EOF from 127.0.0.1:55718 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 11:10:40.059302 30276 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc' of type FILE
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.060484 30276 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.061717 30276 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:40.071903 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:40.062333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.29.145.1:38003)
0811 11:10:40.062400 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.062402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.063573 (+ 1171us) server_negotiation.cc:407] Connection header received
0811 11:10:40.063774 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.063775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.063788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.063845 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.064258 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.064960 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.065934 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.066156 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.066233 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:40.068898 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:40.068899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:40.068899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:40.068908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:40.070504 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.071053 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.071055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.071055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.071129 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.071355 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.071356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.071357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.071476 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:40.071588 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:40.071702 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 11:10:40.071793 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 11:10:40.074718 4655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:40.073565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.29.145.1:38003)
0811 11:10:40.073727 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.073729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.073737 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:40.073757 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.073758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.073767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.074059 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.074641 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55744: BlockingRecv error: recv got EOF from 127.0.0.1:55744 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260811 11:10:40.074934 30276 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786446611237378-30276-0/krb5kdc/krb5cc' of type FILE
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.076817 30276 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:40.078049 30276 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:10:40.086905 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:40.079019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.29.145.1:38003)
0811 11:10:40.079111 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:40.079113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:40.079122 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:40.079148 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:40.079149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:40.079158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:40.079218 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:40.079607 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.080160 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.081066 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:40.081286 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:40.081362 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:40.083905 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:40.083907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:40.083907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:40.083915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:40.085626 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.086156 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.086157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.086157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.086238 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:40.086429 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:40.086430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:40.086430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:40.086526 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:40.086600 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:40.086718 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 11:10:40.086777 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:10:40.087683 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4189
I20260811 11:10:40.097662 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4347
I20260811 11:10:40.107183 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4498
I20260811 11:10:40.114738 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4080
2026-08-11T11:10:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3555 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4671](info): set up 2 sockets
Aug 11 11:10:40 dist-test-slave-69rg krb5kdc[4671](info): commencing operation
krb5kdc: starting...
W20260811 11:10:41.302373 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:41 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab.
Entry for principal oryx/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab.
I20260811 11:10:41.462672 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45883
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab
--principal=oryx/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:45883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:41.565574 4687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:41.565922 4687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:41.565968 4687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:41.567394 4687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:41.567466 4687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:41.567482 4687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:41.567497 4687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:41.567509 4687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:41.569620 4687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:45883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab
--principal=oryx/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:41.570024 4687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:41.570345 4687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:41.576460 4695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:41.576803 4693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:41.576762 4692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:41.576675 4687 server_base.cc:1034] running on GCE node
I20260811 11:10:41.577201 4687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:41.577590 4687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:41.578860 4687 hybrid_clock.cc:648] HybridClock initialized: now 1786446641578815 us; error 67 us; skew 500 ppm
Aug 11 11:10:41 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:41.582405 4687 init.cc:377] Logged in from keytab as oryx/127.29.145.62@KRBTEST.COM (short username oryx)
I20260811 11:10:41.583071 4687 webserver.cc:460] Webserver started at http://127.29.145.62:39221/ using document root <none> and password file <none>
I20260811 11:10:41.583380 4687 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:41.583461 4687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:41.583653 4687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:41.585212 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "dcdecc366cc8437a9b09550699839976"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2c05abb3b53027a833bb9ba5f9e7b551"
server_key_iv: "fc1c6694db8c7695320f7d762c69d859"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.585708 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "dcdecc366cc8437a9b09550699839976"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2c05abb3b53027a833bb9ba5f9e7b551"
server_key_iv: "fc1c6694db8c7695320f7d762c69d859"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.587491 4687 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:41.588668 4704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:41.588946 4687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:41.589085 4687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "dcdecc366cc8437a9b09550699839976"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2c05abb3b53027a833bb9ba5f9e7b551"
server_key_iv: "fc1c6694db8c7695320f7d762c69d859"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.589188 4687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:41.610200 4687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:41.613165 4687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:41.613377 4687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:41.618762 4687 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:45883
I20260811 11:10:41.618820 4767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:45883 every 8 connection(s)
I20260811 11:10:41.619287 4687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:41.620545 4768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:41.620971 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4687
I20260811 11:10:41.621158 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:41.621378 30276 external_mini_cluster.cc:1428] Setting key 062f81999f1a0d821991b18fd3cd9f7b
I20260811 11:10:41.623893 4768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Bootstrap starting.
I20260811 11:10:41.624636 4768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:41.625015 4768 log.cc:824] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:41.625957 4768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: No bootstrap required, opened a new log
I20260811 11:10:41.628275 4768 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } }
I20260811 11:10:41.628465 4768 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:41.628548 4768 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcdecc366cc8437a9b09550699839976, State: Initialized, Role: FOLLOWER
I20260811 11:10:41.628747 4768 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } }
I20260811 11:10:41.628878 4768 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:41.628925 4768 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:41.628979 4768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 11:10:41 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:41.629907 4768 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } }
I20260811 11:10:41.630059 4768 leader_election.cc:302] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcdecc366cc8437a9b09550699839976; no voters:
I20260811 11:10:41.630414 4768 leader_election.cc:288] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:41.630456 4773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:41.630635 4773 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 LEADER]: Becoming Leader. State: Replica: dcdecc366cc8437a9b09550699839976, State: Running, Role: LEADER
I20260811 11:10:41.630788 4768 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:41.630810 4773 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } }
I20260811 11:10:41.632022 4774 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } }
I20260811 11:10:41.632136 4774 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:41.632436 4775 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcdecc366cc8437a9b09550699839976. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } }
I20260811 11:10:41.632503 4775 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:41.632480 4785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:41.633577 4785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:41.634972 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:41.622155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.29.145.62:45883)
0811 11:10:41.622411 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:41.622424 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:41.622464 (+ 40us) server_negotiation.cc:407] Connection header received
0811 11:10:41.623104 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:41.623113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:41.623269 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:41.623511 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:41.624340 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.625624 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.626583 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.627056 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.627186 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:41.630628 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:41.630634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:41.630634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:41.630653 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:41.633190 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:41.633628 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:41.633633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:41.633633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:41.633757 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:41.634030 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:41.634031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:41.634031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:41.634312 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:41.634475 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:41.634603 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 11:10:41.634754 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
I20260811 11:10:41.639609 4785 catalog_manager.cc:1347] Generated new cluster ID: fbb382d3ab0643c6a70e265c8e6b95d9
I20260811 11:10:41.639717 4785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:41.656437 4785 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:41.657296 4785 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:41.670008 4785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Generated new TSK 0
I20260811 11:10:41.670313 4785 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
I20260811 11:10:41.709317 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:41.806784 4795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:41.807017 4795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:41.807049 4795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:41.808681 4795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:41.808749 4795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:41.808777 4795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:41.810504 4795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:41.810792 4795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:41.811086 4795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:41.816953 4802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:41.817014 4795 server_base.cc:1034] running on GCE node
W20260811 11:10:41.817027 4806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:41.816978 4804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:41.817407 4795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:41.817678 4795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:41.818851 4795 hybrid_clock.cc:648] HybridClock initialized: now 1786446641818847 us; error 58 us; skew 500 ppm
Aug 11 11:10:41 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:41.821882 4795 init.cc:377] Logged in from keytab as oryx/127.29.145.1@KRBTEST.COM (short username oryx)
I20260811 11:10:41.822454 4795 webserver.cc:460] Webserver started at http://127.29.145.1:40667/ using document root <none> and password file <none>
I20260811 11:10:41.822719 4795 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:41.822793 4795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:41.822958 4795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:41.824350 4795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2a6580543937bcfc454927fd24851c8c"
server_key_iv: "0b46a14aa294ce7246f49c817c2caaee"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.824805 4795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2a6580543937bcfc454927fd24851c8c"
server_key_iv: "0b46a14aa294ce7246f49c817c2caaee"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.826470 4795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 11:10:41.827375 4814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:41.827594 4795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:41.827718 4795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2a6580543937bcfc454927fd24851c8c"
server_key_iv: "0b46a14aa294ce7246f49c817c2caaee"
server_key_version: "encryptionkey@0"
I20260811 11:10:41.827800 4795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:41.842144 4795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:41.844808 4795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:41.845016 4795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:41.845322 4795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:41.845713 4795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:41.845777 4795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:41.845824 4795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:41.845894 4795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:41.853019 4795 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:46827
I20260811 11:10:41.853075 4947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:46827 every 8 connection(s)
I20260811 11:10:41.853513 4795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:41.857057 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4795
I20260811 11:10:41.857169 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:41.857399 30276 external_mini_cluster.cc:1428] Setting key 004faa7e131d96d66f630dd70eaf36a6
Aug 11 11:10:41 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:41.866786 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:41.854559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48311 (local address 127.29.145.62:45883)
0811 11:10:41.854671 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:41.854674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:41.854835 (+ 161us) server_negotiation.cc:407] Connection header received
0811 11:10:41.855494 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:41.855495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:41.855508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:41.855566 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:41.856764 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.857657 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.859180 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.859449 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.859519 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:41.862659 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:41.862662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:41.862662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:41.862675 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:41.864709 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:41.865374 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:41.865375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:41.865376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:41.865507 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:41.865882 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:41.865883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:41.865883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:41.866109 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:41.866204 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:41.866581 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 11:10:41.866680 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 11:10:41.867013 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:41.854468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:48311)
0811 11:10:41.854681 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:41.854729 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:41.855358 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:41.855638 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:41.855642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:41.856384 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:41.856656 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:41.856662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.857772 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.857774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:41.859075 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:41.859081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:41.859547 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:41.859548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:41.859712 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:41.860409 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:41.860422 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:41.862539 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:41.864812 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:41.864819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:41.864822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:41.865200 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:41.865593 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:41.865594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:41.865595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:41.865784 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:41.866251 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:41.866260 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:41.866422 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 11:10:41.866541 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 11:10:41.866733 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
I20260811 11:10:41.867724 4948 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:41.867868 4948 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:41.868130 4948 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:41.868870 4724 ts_manager.cc:194] Registered new tserver with Master: 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:41.870059 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:48311
WARNING: no policy specified for oryx/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
I20260811 11:10:41.894939 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:41.993258 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:41.993521 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:41.993559 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:41.994776 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:41.994848 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:41.994894 4955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:41.996397 4955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:41.996688 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:41.996945 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.002558 4962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.002552 4966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.002686 4964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.002781 4955 server_base.cc:1034] running on GCE node
I20260811 11:10:42.003077 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.003314 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.004500 4955 hybrid_clock.cc:648] HybridClock initialized: now 1786446642004478 us; error 45 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.007396 4955 init.cc:377] Logged in from keytab as oryx/127.29.145.2@KRBTEST.COM (short username oryx)
I20260811 11:10:42.007997 4955 webserver.cc:460] Webserver started at http://127.29.145.2:34979/ using document root <none> and password file <none>
I20260811 11:10:42.008200 4955 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.008244 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.008343 4955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:42.009404 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "c48342c3ef348510203e77e3e802f602"
server_key_iv: "fbeb7d6cc96157fd1e4891f4ea68729f"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.009743 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "c48342c3ef348510203e77e3e802f602"
server_key_iv: "fbeb7d6cc96157fd1e4891f4ea68729f"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.011194 4955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:42.012009 4973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.012310 4955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:42.012408 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "c48342c3ef348510203e77e3e802f602"
server_key_iv: "fbeb7d6cc96157fd1e4891f4ea68729f"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.012468 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.025486 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.027642 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:42.027776 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.027977 4955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:42.028273 4955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:42.028307 4955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.028329 4955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:42.028343 4955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.035084 4955 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35361
I20260811 11:10:42.035151 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35361 every 8 connection(s)
I20260811 11:10:42.035514 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:42.042829 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 4955
I20260811 11:10:42.042966 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:42.043244 30276 external_mini_cluster.cc:1428] Setting key eea968e9c51eaf3a0a145dc9c228dc28
I20260811 11:10:42.046504 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.036391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:55189 (local address 127.29.145.62:45883)
0811 11:10:42.036447 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.036450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.036723 (+ 273us) server_negotiation.cc:407] Connection header received
0811 11:10:42.037221 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.037222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.037235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.037307 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.038481 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.039049 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.040182 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.040386 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.040455 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.043124 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.043126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.043126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.043142 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.044919 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.045397 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.045398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.045398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.045466 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.045741 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.045743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.045743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.045915 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.045998 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.046307 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.046409 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:10:42.046710 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.036336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:55189)
0811 11:10:42.036527 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.036574 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.037084 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.037393 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.037397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.038161 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.038378 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.038383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.039174 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.039175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.040046 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.040048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.040497 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.040499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.040621 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.041302 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.041311 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.042982 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.045016 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.045024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.045027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.045307 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.045542 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.045543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.045543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.045666 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.046000 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.046008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.046156 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.046273 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.046445 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260811 11:10:42.047259 5104 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:42.047379 5104 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:42.047590 5104 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:42.048112 4724 ts_manager.cc:194] Registered new tserver with Master: 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2:35361)
I20260811 11:10:42.048682 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:55189
WARNING: no policy specified for oryx/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
I20260811 11:10:42.073050 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:42.164419 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:42.164671 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:42.164710 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:42.166188 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:42.166271 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:42.166319 5110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:42.168056 5110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:45883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:42.168365 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:42.168625 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.174091 5117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.174094 5118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.174106 5120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.174374 5110 server_base.cc:1034] running on GCE node
I20260811 11:10:42.174618 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.174872 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.176097 5110 hybrid_clock.cc:648] HybridClock initialized: now 1786446642176052 us; error 73 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.178843 5110 init.cc:377] Logged in from keytab as oryx/127.29.145.3@KRBTEST.COM (short username oryx)
I20260811 11:10:42.179330 5110 webserver.cc:460] Webserver started at http://127.29.145.3:40743/ using document root <none> and password file <none>
I20260811 11:10:42.179600 5110 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.179651 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.179775 5110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:42.180879 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "35c08d229e4b33fb4cb980e950a41e9a"
server_key_iv: "2a873db3f0669a35b38bace38e154f2e"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.181257 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "35c08d229e4b33fb4cb980e950a41e9a"
server_key_iv: "2a873db3f0669a35b38bace38e154f2e"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.182937 5110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:42.183969 5132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.184365 5110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.184465 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "35c08d229e4b33fb4cb980e950a41e9a"
server_key_iv: "2a873db3f0669a35b38bace38e154f2e"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.184571 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.202893 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.205090 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:42.205224 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.205547 5110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:42.205906 5110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:42.205946 5110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.205971 5110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:42.205982 5110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.212761 5110 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:33451
I20260811 11:10:42.212805 5257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:33451 every 8 connection(s)
I20260811 11:10:42.213184 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:42.219220 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 5110
I20260811 11:10:42.219307 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:42.219508 30276 external_mini_cluster.cc:1428] Setting key 1feaa708b46119d16693aac37a8e34b0
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:42.223428 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.214055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:54177 (local address 127.29.145.62:45883)
0811 11:10:42.214145 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.214148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.214382 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:10:42.214878 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.214879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.214890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.214943 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.216085 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.216684 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.217868 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.218100 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.218199 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.220811 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.220812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.220812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.220818 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.222209 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.222583 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.222584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.222584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.222640 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.222862 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.222863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.222863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.222985 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.223034 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.223289 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.223355 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:10:42.223629 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.214018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:54177)
0811 11:10:42.214251 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.214296 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.214793 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.214998 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.215002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.215730 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.215963 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.215968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.216800 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.216802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.217707 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.217710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.218207 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.218208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.218353 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.218952 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.218960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.220725 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.222288 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.222294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.222296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.222515 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.222699 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.222700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.222700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.222808 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.223044 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.223051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.223172 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.223262 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.223380 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260811 11:10:42.224187 5259 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:42.224301 5259 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:42.224454 5259 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:42.224905 4724 ts_manager.cc:194] Registered new tserver with Master: 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:42.225452 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:54177
I20260811 11:10:42.231611 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.145.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.61.keytab.
Entry for principal oryx/127.29.145.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.61.keytab.
I20260811 11:10:42.261777 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:37055
--webserver_interface=127.29.145.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.61.keytab
--principal=oryx/127.29.145.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:45883,127.29.145.61:37055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:42.347864 5266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:42.348073 5266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:42.348096 5266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:42.349177 5266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:42.349212 5266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:42.349227 5266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:42.349239 5266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:42.349251 5266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:42.350646 5266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:45883,127.29.145.61:37055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.61.keytab
--principal=oryx/127.29.145.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:37055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.145.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
W20260811 11:10:42.350751 5266 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.29.145.62:45883,127.29.145.61:37055'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 11:10:42.350860 5266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:42.351049 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.356767 5271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.356851 5272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.356781 5274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.357072 5266 server_base.cc:1034] running on GCE node
I20260811 11:10:42.357393 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.357728 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.358953 5266 hybrid_clock.cc:648] HybridClock initialized: now 1786446642358946 us; error 53 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.362133 5266 init.cc:377] Logged in from keytab as oryx/127.29.145.61@KRBTEST.COM (short username oryx)
I20260811 11:10:42.362656 5266 webserver.cc:460] Webserver started at http://127.29.145.61:41399/ using document root <none> and password file <none>
I20260811 11:10:42.362911 5266 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.362962 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.363106 5266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:42.364324 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/instance:
uuid: "1f0dcbba5423443b8c6c887bec4c1218"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "a9ac1eb4c2406b3d81370507345c1004"
server_key_iv: "e4075f5879b315899d8a4e285cf4ad23"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.364759 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal/instance:
uuid: "1f0dcbba5423443b8c6c887bec4c1218"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "a9ac1eb4c2406b3d81370507345c1004"
server_key_iv: "e4075f5879b315899d8a4e285cf4ad23"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.366410 5266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:42.367332 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.367519 5266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.367638 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "1f0dcbba5423443b8c6c887bec4c1218"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "a9ac1eb4c2406b3d81370507345c1004"
server_key_iv: "e4075f5879b315899d8a4e285cf4ad23"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.367707 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.382833 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.385309 5266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:42.385430 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.389925 5266 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.61:37055
I20260811 11:10:42.389979 5338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.61:37055 every 8 connection(s)
I20260811 11:10:42.390340 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
I20260811 11:10:42.391333 5341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:42.393138 5341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:10:42.398025 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 5266
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.61@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:42.403717 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.393923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.29.145.62:45883)
0811 11:10:42.393990 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.393992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.394219 (+ 227us) server_negotiation.cc:407] Connection header received
0811 11:10:42.394685 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.394685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.394695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.394756 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.395333 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.395853 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.396852 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.397113 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.397239 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.400081 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.400082 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.400083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.400091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.401763 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.402281 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.402282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.402283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.402355 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.402818 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.402819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.402820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.402993 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.403051 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.403521 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.403626 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:10:42.403927 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.393914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48600)
0811 11:10:42.394085 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.394127 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.394592 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.394814 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.394817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.395059 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.395250 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.395254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.395945 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.395946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.396762 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.396765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.397249 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.397272 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.397460 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.398204 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.398214 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.399987 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.401935 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.401941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.401943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.402181 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.402429 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.402430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.402431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.402679 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.403088 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.403093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.403280 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.403468 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.403677 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
I20260811 11:10:42.404740 5341 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:10:42.408228 5345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.398715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.29.145.61:37055)
0811 11:10:42.398909 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.398915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.398937 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:10:42.399003 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.399006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.399029 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.399132 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.399785 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.400820 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.401736 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.402149 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.402234 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.404803 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.404808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.404809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.404816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.406864 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.407269 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.407273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.407273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.407340 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.407553 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.407553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.407554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.407753 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.407919 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.408027 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.408115 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":77,"threads_started":1}
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.61@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
I20260811 11:10:42.412452 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.405036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33638)
0811 11:10:42.405094 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.405106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.405148 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.405447 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.405448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.405633 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.405696 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.405698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.406553 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.406554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.407207 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.407209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.407694 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.407695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.407775 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.408201 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.408206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.409507 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.411578 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.411578 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.411579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.411739 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.411970 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.411971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.411971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.412023 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.412236 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.412236 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.412270 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.412321 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.412394 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:10:42.412637 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.405112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.29.145.61:37055)
0811 11:10:42.405294 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.405296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.405304 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:42.405323 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.405324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.405334 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.405408 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.405793 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.406467 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.407318 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.407625 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.407674 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.409614 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.409615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.409616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.409622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.411497 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.411841 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.411842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.411842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.411901 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.412064 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.412064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.412064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.412181 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.412233 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.412339 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.412442 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260811 11:10:42.413991 5341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Bootstrap starting.
I20260811 11:10:42.414585 5341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:42.414831 5341 log.cc:824] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:42.415733 5341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: No bootstrap required, opened a new log
I20260811 11:10:42.417225 5341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } }
I20260811 11:10:42.417353 5341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:42.417378 5341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0dcbba5423443b8c6c887bec4c1218, State: Initialized, Role: FOLLOWER
I20260811 11:10:42.417467 5341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } }
I20260811 11:10:42.417714 5341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:42.417686 5348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } } }
I20260811 11:10:42.417865 5348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:10:42.419559 5359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:10:42.419617 5359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:10:42.421298 5266 master_runner.cc:428] Detected that this master 1f0dcbba5423443b8c6c887bec4c1218 is joining an existing cluster
I20260811 11:10:42.421489 5266 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.145.61:37055
I20260811 11:10:42.429503 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.422916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.29.145.61:37055)
0811 11:10:42.422956 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.422958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.423077 (+ 119us) server_negotiation.cc:407] Connection header received
0811 11:10:42.423148 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.423149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.423158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.423207 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.423579 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.424316 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425091 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.425325 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425408 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.426679 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.426680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.426680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.426686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.428278 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428640 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.428641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.428641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.428693 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428920 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.428921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.428922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.429052 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.429115 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.429285 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.429411 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":22}
I20260811 11:10:42.429503 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.422888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33646)
0811 11:10:42.423002 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.423011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.423101 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.423242 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.423243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.423426 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.423495 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.423497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.424404 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.424404 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.425005 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.425006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.425472 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.425529 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.426063 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.426067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.426584 (+ 517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.428383 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.428384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.428565 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.428787 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.428788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.428843 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.429136 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.429136 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.429177 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.429251 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.429358 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":39,"thread_start_us":71,"threads_started":1}
I20260811 11:10:42.429885 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.422662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.29.145.62:45883)
0811 11:10:42.422710 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.422712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.422879 (+ 167us) server_negotiation.cc:407] Connection header received
0811 11:10:42.422986 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.422987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.423000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.423030 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.423519 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.424320 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425387 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.425687 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425796 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.426682 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.426684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.426684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.426692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.428603 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428983 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.428984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.428985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.429038 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.429296 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.429298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.429298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.429433 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.429481 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.429690 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.429768 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:10:42.430439 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.422623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48608)
0811 11:10:42.422799 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.422812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.422908 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.423105 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.423106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.423349 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.423429 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.423431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.424405 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.424407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.425282 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.425284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.425800 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.425802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.425900 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.426211 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.426215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.426591 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.428701 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.428702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.428702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.428917 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.429112 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.429113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.429113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.429192 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.429532 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.429533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.429573 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.430207 (+ 634us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.430343 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.62@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
I20260811 11:10:42.439966 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.432865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.29.145.61:37055)
0811 11:10:42.432900 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.432902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.433065 (+ 163us) server_negotiation.cc:407] Connection header received
0811 11:10:42.433168 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.433169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.433180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.433228 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.433662 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.434183 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.434963 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.435167 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.435259 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.437362 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.437363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.437364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.437370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.438746 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.439194 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.439195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.439195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.439244 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.439458 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.439459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.439459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.439553 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.439603 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.439842 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.439906 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:10:42.440011 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.432825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33654)
0811 11:10:42.432969 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.433004 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.433100 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.433231 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.433235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.433519 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.433567 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.433569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.434277 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.434278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.434873 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.434876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.435256 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.435257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.435338 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.435818 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.435824 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.437261 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.438836 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.438844 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.438846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.439110 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.439325 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.439326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.439327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.439395 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.439615 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.439623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.439678 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.439771 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.439895 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260811 11:10:42.440661 4724 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.145.61:37055: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218"
member_type: NON_VOTER
last_known_addr {
host: "127.29.145.61"
port: 37055
}
attrs {
promote: true
}
}
dest_uuid: "dcdecc366cc8437a9b09550699839976"
cas_config_opid_index: -1
I20260811 11:10:42.441084 4724 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } }
I20260811 11:10:42.441608 5369 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } }
I20260811 11:10:42.442000 5311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:42.442699 5369 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.145.61:37055
I20260811 11:10:42.442950 5311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 11:10:42.443007 5266 master.cc:537] Master@127.29.145.61:37055 shutting down...
I20260811 11:10:42.442986 5368 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } }
I20260811 11:10:42.443068 5368 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:42.445044 5368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:10:42.445907 4708 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 11:10:42.446061 5266 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:42.446166 5266 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:42.446215 5266 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: stopping tablet replica
W20260811 11:10:42.446352 4708 consensus_peers.cc:487] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 -> Peer 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61:37055): Couldn't send request to peer 1f0dcbba5423443b8c6c887bec4c1218. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 11:10:42.449424 5266 master.cc:552] Master@127.29.145.61:37055 shutdown complete.
I20260811 11:10:42.449474 5266 master_runner.cc:305] Clearing existing system tablet
I20260811 11:10:42.449606 5266 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.449656 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.450384 5266 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.450986 5374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.451293 5266 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.451357 5266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.451382 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "1f0dcbba5423443b8c6c887bec4c1218"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "a9ac1eb4c2406b3d81370507345c1004"
server_key_iv: "e4075f5879b315899d8a4e285cf4ad23"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.452154 5266 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:42.453848 5266 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 11:10:42.453918 5266 log.cc:1196] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 11:10:42.454121 5266 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Deleting consensus metadata
I20260811 11:10:42.454243 5266 master_runner.cc:315] Copying system tablet from 127.29.145.62:45883
I20260811 11:10:42.454861 5266 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: tablet copy: Beginning tablet copy session from remote peer at address 127.29.145.62:45883
I20260811 11:10:42.461540 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.455224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.29.145.62:45883)
0811 11:10:42.455296 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.455299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.455423 (+ 124us) server_negotiation.cc:407] Connection header received
0811 11:10:42.455454 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.455455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.455466 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.455537 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.455860 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.456644 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.457370 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.457681 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.457753 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.458826 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.458827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.458827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.458835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.460377 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.460740 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.460742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.460742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.460803 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.461011 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.461012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.461013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.461132 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.461213 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.461367 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.461446 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:10:42.461751 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.455168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48618)
0811 11:10:42.455350 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.455360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.455413 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.455539 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.455540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.455746 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.455789 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.455791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.456744 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.456745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.457285 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.457287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.457765 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.457766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.457886 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.458418 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.458423 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.458735 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.460484 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.460484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.460485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.460659 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.460879 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.460880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.460880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.460942 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.461205 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.461206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.461251 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.461340 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.461447 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260811 11:10:42.462215 4746 tablet_copy_service.cc:140] P dcdecc366cc8437a9b09550699839976: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1f0dcbba5423443b8c6c887bec4c1218 ({username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:48618)
I20260811 11:10:42.462329 4746 tablet_copy_service.cc:161] P dcdecc366cc8437a9b09550699839976: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1f0dcbba5423443b8c6c887bec4c1218 at {username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:48618: session id = 1f0dcbba5423443b8c6c887bec4c1218-00000000000000000000000000000000
I20260811 11:10:42.463091 4746 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 11:10:42.463799 5266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 11:10:42.465094 5266 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: tablet copy: Starting download of 0 data blocks...
I20260811 11:10:42.465245 5266 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: tablet copy: Starting download of 1 WAL segments...
I20260811 11:10:42.466320 5266 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 11:10:42.467242 4746 tablet_copy_service.cc:342] P dcdecc366cc8437a9b09550699839976: Request end of tablet copy session 1f0dcbba5423443b8c6c887bec4c1218-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:48618
I20260811 11:10:42.467319 4746 tablet_copy_service.cc:433] P dcdecc366cc8437a9b09550699839976: ending tablet copy session 1f0dcbba5423443b8c6c887bec4c1218-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1f0dcbba5423443b8c6c887bec4c1218
I20260811 11:10:42.470193 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.471418 5386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.471490 5382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.471571 5383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.471748 5266 server_base.cc:1034] running on GCE node
I20260811 11:10:42.471830 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.471963 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.473093 5266 hybrid_clock.cc:648] HybridClock initialized: now 1786446642473089 us; error 37 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.474426 5266 init.cc:377] Logged in from keytab as oryx/127.29.145.61@KRBTEST.COM (short username oryx)
I20260811 11:10:42.474736 5266 webserver.cc:460] Webserver started at http://127.29.145.61:42237/ using document root <none> and password file <none>
I20260811 11:10:42.474874 5266 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.474921 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.475528 5266 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:42.476084 5395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.476270 5266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:42.476329 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "1f0dcbba5423443b8c6c887bec4c1218"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "a9ac1eb4c2406b3d81370507345c1004"
server_key_iv: "e4075f5879b315899d8a4e285cf4ad23"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.476401 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.486893 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.488693 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.491936 5266 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.61:37055
I20260811 11:10:42.492213 5457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.61:37055 every 8 connection(s)
I20260811 11:10:42.492380 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
I20260811 11:10:42.493280 5458 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:10:42.493765 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Bootstrap starting.
I20260811 11:10:42.496085 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:10:42.496326 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Bootstrap complete.
I20260811 11:10:42.496515 5458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } }
I20260811 11:10:42.496606 5458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1f0dcbba5423443b8c6c887bec4c1218, State: Initialized, Role: LEARNER
I20260811 11:10:42.496658 5458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } }
I20260811 11:10:42.496829 5458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:42.496814 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } }
I20260811 11:10:42.496896 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:10:42.498250 5475 catalog_manager.cc:1259] Loaded cluster ID: fbb382d3ab0643c6a70e265c8e6b95d9
I20260811 11:10:42.498319 5475 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: loading cluster ID for follower catalog manager: success
I20260811 11:10:42.499343 5475 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: acquiring CA information for follower catalog manager: success
I20260811 11:10:42.499626 5475 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:10:42.517040 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.510152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.29.145.61:37055)
0811 11:10:42.510298 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.510300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.510307 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:10:42.510329 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.510329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.510339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.510423 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.511010 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.511707 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.512466 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.512688 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.512772 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.514143 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.514145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.514145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.514152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.515695 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.516089 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.516090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.516090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.516159 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.516436 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.516436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.516437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.516541 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.516649 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.516801 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.516893 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.29.145.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.60.keytab.
Entry for principal oryx/127.29.145.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.60.keytab.
I20260811 11:10:42.549090 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:44557
--webserver_interface=127.29.145.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.60.keytab
--principal=oryx/127.29.145.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:42.636361 5479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:42.636570 5479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:42.636592 5479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:42.637732 5479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:42.637771 5479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:42.637791 5479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:42.637805 5479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:42.637899 5479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:42.639425 5479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.60.keytab
--principal=oryx/127.29.145.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:44557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.145.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:42.639678 5479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:42.639919 5479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.645102 5486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.645148 5489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.645110 5484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.645437 5479 server_base.cc:1034] running on GCE node
I20260811 11:10:42.645753 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.646068 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.647249 5479 hybrid_clock.cc:648] HybridClock initialized: now 1786446642647232 us; error 44 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.649778 5479 init.cc:377] Logged in from keytab as oryx/127.29.145.60@KRBTEST.COM (short username oryx)
I20260811 11:10:42.650288 5479 webserver.cc:460] Webserver started at http://127.29.145.60:44755/ using document root <none> and password file <none>
I20260811 11:10:42.650522 5479 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.650592 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.650755 5479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:42.652061 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/instance:
uuid: "e65b9e270550410b808b9cda6e3145b6"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "4a6b4dec39a96b75f8531312ca02f6d1"
server_key_iv: "c409440ecbc558e0726ae8464e169ab1"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.652487 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal/instance:
uuid: "e65b9e270550410b808b9cda6e3145b6"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "4a6b4dec39a96b75f8531312ca02f6d1"
server_key_iv: "c409440ecbc558e0726ae8464e169ab1"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.653997 5479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:10:42.654868 5497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.655057 5479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.655174 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "e65b9e270550410b808b9cda6e3145b6"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "4a6b4dec39a96b75f8531312ca02f6d1"
server_key_iv: "c409440ecbc558e0726ae8464e169ab1"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.655292 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.678999 5479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.681274 5479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:42.681473 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.685858 5479 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.60:44557
I20260811 11:10:42.685869 5557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.60:44557 every 8 connection(s)
I20260811 11:10:42.686305 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
I20260811 11:10:42.687300 5558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:42.688987 5558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:10:42.694976 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 5479
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.60@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:42.698946 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.689854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.29.145.62:45883)
0811 11:10:42.689913 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.689916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.690141 (+ 225us) server_negotiation.cc:407] Connection header received
0811 11:10:42.690680 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.690681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.690696 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.690758 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.691428 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.692015 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.693101 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.693296 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.693375 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.695814 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.695815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.695815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.695822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.697423 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.697901 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.697902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.697903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.697972 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.698341 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.698342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.698342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.698454 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.698505 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.698820 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.698883 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:10:42.699183 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.689676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48630)
0811 11:10:42.689970 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.690007 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.690550 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.690843 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.690847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.691113 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.691317 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.691321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.692109 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.692111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.692969 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.692972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.693395 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.693396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.693526 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.694156 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.694165 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.695670 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.697499 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.697504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.697506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.697799 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.698059 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.698059 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.698059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.698270 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.698521 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.698527 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.698677 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.698777 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.698947 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":149,"threads_started":1}
I20260811 11:10:42.699928 5558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.60@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
I20260811 11:10:42.705070 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.695841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.29.145.60:44557)
0811 11:10:42.696027 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.696032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.696056 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:10:42.696080 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.696082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.696112 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.696208 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.696874 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.697879 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.698922 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.699349 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.699434 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.702067 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.702074 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.702075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.702081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.703814 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.704190 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.704193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.704194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.704258 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.704490 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.704491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.704491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.704643 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.704796 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.704870 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.704954 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260811 11:10:42.707288 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.700185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33686)
0811 11:10:42.700938 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.700948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.701001 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.701132 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.701133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.701279 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.701340 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.701341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.702127 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.702128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.702627 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.702629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.703025 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.703026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.703103 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.703454 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.703458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.704641 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.706352 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.706352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.706353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.706509 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.706744 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.706745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.706745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.706809 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.707073 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.707074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.707109 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.707163 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.707231 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733}
I20260811 11:10:42.707315 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.700268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.29.145.61:37055)
0811 11:10:42.700311 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.700314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.701001 (+ 687us) server_negotiation.cc:407] Connection header received
0811 11:10:42.701065 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.701065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.701076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.701123 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.701410 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.702023 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.702724 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.702944 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.703027 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.704735 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.704737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.704737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.704747 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.706275 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.706601 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.706602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.706603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.706669 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.706893 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.706894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.706894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.707003 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.707057 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.707183 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.707256 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:42.707726 5558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.60@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
I20260811 11:10:42.713517 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.708005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45006)
0811 11:10:42.708056 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.708067 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.708117 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.708213 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.708214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.708324 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.708351 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.708352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.708898 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.708899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.709408 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.709410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.709763 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.709764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.709806 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.710227 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.710232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.711302 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.712604 (+ 1302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.712605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.712605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.712757 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.712939 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.712939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.712940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.712988 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.713260 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.713261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.713301 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.713370 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.713459 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 11:10:42.713591 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.708046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.29.145.60:44557)
0811 11:10:42.708080 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.708082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.708110 (+ 28us) server_negotiation.cc:407] Connection header received
0811 11:10:42.708162 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.708162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.708170 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.708211 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.708399 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.708837 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.709495 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.709694 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.709753 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.711378 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.711379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.711379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.711385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.712533 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.712829 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.712830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.712831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.712883 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.713027 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.713028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.713028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.713152 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.713201 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.713404 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.713505 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":17}
I20260811 11:10:42.715225 5558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Bootstrap starting.
I20260811 11:10:42.716185 5558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:42.716470 5558 log.cc:824] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:42.717257 5558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: No bootstrap required, opened a new log
I20260811 11:10:42.718802 5558 raft_consensus.cc:348] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } }
I20260811 11:10:42.718927 5558 raft_consensus.cc:374] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:42.718950 5558 raft_consensus.cc:729] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65b9e270550410b808b9cda6e3145b6, State: Initialized, Role: FOLLOWER
I20260811 11:10:42.719091 5558 consensus_queue.cc:260] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } }
I20260811 11:10:42.719425 5558 sys_catalog.cc:564] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:42.719393 5563 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } } }
I20260811 11:10:42.719503 5563 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:10:42.721289 5576 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:10:42.721346 5576 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:10:42.723906 5479 master_runner.cc:428] Detected that this master e65b9e270550410b808b9cda6e3145b6 is joining an existing cluster
I20260811 11:10:42.723977 5479 master_runner.cc:432] Initiating AddMaster RPC to add 127.29.145.60:44557
I20260811 11:10:42.750108 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.724989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.29.145.60:44557)
0811 11:10:42.725038 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.725041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.725467 (+ 426us) server_negotiation.cc:407] Connection header received
0811 11:10:42.725508 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.725509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.725518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.725573 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.725999 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.726675 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.727748 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728029 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728064 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.746806 (+ 18742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.746808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.746808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.746818 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.748776 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749208 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.749209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.749209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.749266 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749448 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.749448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.749449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.749595 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.749639 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.749885 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.749987 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:42.750699 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.724916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48642)
0811 11:10:42.725060 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.725072 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.725158 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.725396 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.725397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.725652 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.725720 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.725722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.726725 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.726726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.727693 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.727696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728212 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.728273 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.747074 (+ 18801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.747080 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.747619 (+ 539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.749528 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.749530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.749746 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.750008 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.750008 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.750009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.750068 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.750357 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.750358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.750404 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.750468 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.750586 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":49,"thread_start_us":83,"threads_started":1}
I20260811 11:10:42.750948 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.724994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48642 (local address 127.29.145.62:45883)
0811 11:10:42.725046 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.725048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.725129 (+ 81us) server_negotiation.cc:407] Connection header received
0811 11:10:42.725257 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.725259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.725270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.725306 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.725807 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.726609 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.727795 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728095 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728155 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.747757 (+ 19602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.747770 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.747770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.747780 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.749421 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749871 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.749872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.749872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.749934 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.750133 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.750133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.750134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.750267 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.750330 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.750755 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.750857 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:10:42.751058 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.725177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.29.145.61:37055)
0811 11:10:42.725217 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.725218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.725964 (+ 746us) server_negotiation.cc:407] Connection header received
0811 11:10:42.726068 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.726069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.726080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.726109 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.726521 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.727245 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728219 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728492 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728527 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.747340 (+ 18813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.747341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.747342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.747352 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.749719 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.750173 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.750174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.750175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.750243 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.750466 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.750467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.750467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.750596 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.750650 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.750880 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.750962 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:10:42.751150 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.725143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45008)
0811 11:10:42.725387 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.725397 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.725455 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.725600 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.725601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.725788 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.725905 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.725908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.726755 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.726756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.727662 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.727664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728160 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.728223 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.728719 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.728724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.746668 (+ 17944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.748872 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.748873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.748875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.749107 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.749331 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749331 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.749332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.749383 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.749682 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.749683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.749746 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.750572 (+ 826us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.750692 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":41,"thread_start_us":71,"threads_started":1}
I20260811 11:10:42.751430 5585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.725257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33696)
0811 11:10:42.725906 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.725917 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.725989 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.726172 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.726173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.726376 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.726433 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.726435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.727338 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.727338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.728130 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.728133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.728633 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.728634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.728700 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.729055 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.729058 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.746948 (+ 17890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.749808 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.749809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.749810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.750081 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.750330 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.750331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.750332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.750398 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.750701 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.750702 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.750745 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.751172 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.751298 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":66,"threads_started":1}
I20260811 11:10:42.756982 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4795
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446641, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.62@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
I20260811 11:10:42.767186 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.754200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.29.145.60:44557)
0811 11:10:42.754254 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.754256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.754265 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:42.754343 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.754343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.754353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.754402 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.754719 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.755197 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.755906 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.756174 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.756234 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.762782 (+ 6548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.762784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.762784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.762793 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.765739 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.766251 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.766252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.766253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.766320 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.766573 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.766574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.766575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.766729 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.766802 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.767002 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.767089 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:10:42.767243 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.754175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45010)
0811 11:10:42.754218 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.754229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.754278 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.754404 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.754405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.754571 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.754625 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.754627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.755278 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.755279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.755815 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.755817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.756233 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.756234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.756311 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.756766 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.756771 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.762664 (+ 5893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.765879 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.765881 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.765882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.766155 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.766406 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.766407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.766408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.766493 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.766812 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.766814 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.766877 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.766956 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.767094 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 11:10:42.769649 4724 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.29.145.60:44557: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e65b9e270550410b808b9cda6e3145b6"
member_type: NON_VOTER
last_known_addr {
host: "127.29.145.60"
port: 44557
}
attrs {
promote: true
}
}
dest_uuid: "dcdecc366cc8437a9b09550699839976"
cas_config_opid_index: 5
I20260811 11:10:42.769846 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:46827
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:45883
--webserver_port=40667
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:10:42.770012 4724 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } }
I20260811 11:10:42.770781 5587 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e65b9e270550410b808b9cda6e3145b6 (127.29.145.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.770982 5529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:42.772003 5529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 11:10:42.772279 5587 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.29.145.60:44557
I20260811 11:10:42.772578 5479 master.cc:537] Master@127.29.145.60:44557 shutting down...
I20260811 11:10:42.774298 5369 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.774405 5369 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:42.776345 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.772298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33698)
0811 11:10:42.772396 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.772407 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.772444 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.772745 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.772747 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.772748 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:42.772794 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.772795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.773805 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.773807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.775030 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.775034 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.776025 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.776027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.776112 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.776138 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.776198 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.776252 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 11:10:42.776576 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.772525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.29.145.61:37055)
0811 11:10:42.772567 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.772569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.772576 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:10:42.772602 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.772603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.772627 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.772683 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:42.772923 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.773694 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.775152 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.775815 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.775894 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.775985 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.776454 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.776504 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:42.776661 5587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:42.776813 5479 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:42.776839 5428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 LEARNER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 11:10:42.776901 5479 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:42.776949 5479 tablet_replica.cc:330] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: stopping tablet replica
I20260811 11:10:42.777500 5587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 11:10:42.777724 4706 consensus_peers.cc:487] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 -> Peer e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557): Couldn't send request to peer e65b9e270550410b808b9cda6e3145b6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 11:10:42.778187 5428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e65b9e270550410b808b9cda6e3145b6 (127.29.145.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.779515 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcdecc366cc8437a9b09550699839976. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.779608 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:10:42.779651 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: NON_VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: true } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.779706 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:10:42.780373 5369 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: attempting to promote NON_VOTER 1f0dcbba5423443b8c6c887bec4c1218 to VOTER
I20260811 11:10:42.780601 5369 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } }
I20260811 11:10:42.781191 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.781052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45012)
0811 11:10:42.781109 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.781160 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:44557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:10:42.781234 5428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 LEARNER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 11:10:42.781392 5587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 11:10:42.781513 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.781427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45024)
0811 11:10:42.781464 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.781487 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:44557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 11:10:42.781579 4706 consensus_peers.cc:487] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 -> Peer e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557): Couldn't send request to peer e65b9e270550410b808b9cda6e3145b6. Status: Network error: Client connection negotiation failed: client connection to 127.29.145.60:44557: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 11:10:42.782130 5591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.782289 5428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.783324 5590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.783442 5590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:42.783696 5369 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.783825 5369 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:42.783708 5368 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.783978 5368 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:42.790771 5479 master.cc:552] Master@127.29.145.60:44557 shutdown complete.
I20260811 11:10:42.790825 5479 master_runner.cc:305] Clearing existing system tablet
I20260811 11:10:42.790970 5479 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.791021 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.791785 5479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:42.792325 5595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.792495 5479 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.792558 5479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:42.792583 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "e65b9e270550410b808b9cda6e3145b6"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "4a6b4dec39a96b75f8531312ca02f6d1"
server_key_iv: "c409440ecbc558e0726ae8464e169ab1"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.793373 5479 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:42.794811 5479 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 11:10:42.794885 5479 log.cc:1196] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 11:10:42.795090 5479 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Deleting consensus metadata
I20260811 11:10:42.795372 5479 master_runner.cc:315] Copying system tablet from 127.29.145.62:45883
I20260811 11:10:42.796010 5479 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: tablet copy: Beginning tablet copy session from remote peer at address 127.29.145.62:45883
I20260811 11:10:42.803272 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.796312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:48654)
0811 11:10:42.796456 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.796470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.796534 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.796671 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.796672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.796923 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.796972 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.796974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.798030 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.798032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.798921 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.798924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.799432 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.799433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.799495 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.800018 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.800024 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.800415 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.802244 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.802245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.802245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.802417 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.802624 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.802625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.802625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.802678 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.803001 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.803001 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.803042 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.803100 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.803181 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260811 11:10:42.803309 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.796477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.29.145.62:45883)
0811 11:10:42.796521 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.796524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.796535 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:42.796594 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.796595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.796605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.796666 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.797054 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.797922 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.799020 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.799307 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.799369 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.800512 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.800514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.800514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.800521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.802153 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.802492 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.802493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.802493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.802553 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.802788 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.802789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.802789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.802921 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.802992 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.803145 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.803223 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:42.803498 4746 tablet_copy_service.cc:140] P dcdecc366cc8437a9b09550699839976: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e65b9e270550410b808b9cda6e3145b6 ({username='oryx', principal='oryx/127.29.145.60@KRBTEST.COM'} at 127.0.0.1:48654)
I20260811 11:10:42.803552 4746 tablet_copy_service.cc:161] P dcdecc366cc8437a9b09550699839976: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e65b9e270550410b808b9cda6e3145b6 at {username='oryx', principal='oryx/127.29.145.60@KRBTEST.COM'} at 127.0.0.1:48654: session id = e65b9e270550410b808b9cda6e3145b6-00000000000000000000000000000000
I20260811 11:10:42.804170 4746 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 11:10:42.804661 5479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 11:10:42.805927 5479 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: tablet copy: Starting download of 0 data blocks...
I20260811 11:10:42.806070 5479 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: tablet copy: Starting download of 1 WAL segments...
I20260811 11:10:42.806830 5479 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 11:10:42.807730 4746 tablet_copy_service.cc:342] P dcdecc366cc8437a9b09550699839976: Request end of tablet copy session e65b9e270550410b808b9cda6e3145b6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.29.145.60@KRBTEST.COM'} at 127.0.0.1:48654
I20260811 11:10:42.807788 4746 tablet_copy_service.cc:433] P dcdecc366cc8437a9b09550699839976: ending tablet copy session e65b9e270550410b808b9cda6e3145b6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e65b9e270550410b808b9cda6e3145b6
I20260811 11:10:42.810539 5479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.811869 5603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.811949 5606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.812542 5604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.812744 5479 server_base.cc:1034] running on GCE node
I20260811 11:10:42.812842 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.812984 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.814119 5479 hybrid_clock.cc:648] HybridClock initialized: now 1786446642814100 us; error 39 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.815623 5479 init.cc:377] Logged in from keytab as oryx/127.29.145.60@KRBTEST.COM (short username oryx)
I20260811 11:10:42.815933 5479 webserver.cc:460] Webserver started at http://127.29.145.60:37493/ using document root <none> and password file <none>
I20260811 11:10:42.816069 5479 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.816109 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.816843 5479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.817478 5614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.817627 5479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:42.817677 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "e65b9e270550410b808b9cda6e3145b6"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "4a6b4dec39a96b75f8531312ca02f6d1"
server_key_iv: "c409440ecbc558e0726ae8464e169ab1"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.817735 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.832808 5479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.834722 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.838042 5479 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.60:44557
I20260811 11:10:42.838090 5676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.60:44557 every 8 connection(s)
I20260811 11:10:42.838410 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
I20260811 11:10:42.839319 5677 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:10:42.839864 5677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Bootstrap starting.
I20260811 11:10:42.843305 5677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:10:42.843575 5677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: Bootstrap complete.
I20260811 11:10:42.843789 5677 raft_consensus.cc:348] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } }
I20260811 11:10:42.843889 5677 raft_consensus.cc:729] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e65b9e270550410b808b9cda6e3145b6, State: Initialized, Role: LEARNER
I20260811 11:10:42.843927 5677 consensus_queue.cc:260] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } }
I20260811 11:10:42.844131 5677 sys_catalog.cc:564] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:42.844135 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:42.844395 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:10:42.845654 5694 catalog_manager.cc:1259] Loaded cluster ID: fbb382d3ab0643c6a70e265c8e6b95d9
I20260811 11:10:42.845743 5694 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: loading cluster ID for follower catalog manager: success
I20260811 11:10:42.847107 5694 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: acquiring CA information for follower catalog manager: success
I20260811 11:10:42.847412 5694 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 11:10:42.872980 5588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:42.873193 5588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:42.873217 5588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:42.874461 5588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:42.874503 5588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:42.874531 5588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:42.876104 5588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:46827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=40667
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:42.876353 5588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:42.876600 5588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:42.882730 5705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.882746 5701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:42.882733 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:42.883236 5588 server_base.cc:1034] running on GCE node
I20260811 11:10:42.883450 5588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:42.883704 5588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:42.884878 5588 hybrid_clock.cc:648] HybridClock initialized: now 1786446642884863 us; error 31 us; skew 500 ppm
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:42.887895 5588 init.cc:377] Logged in from keytab as oryx/127.29.145.1@KRBTEST.COM (short username oryx)
I20260811 11:10:42.888492 5588 webserver.cc:460] Webserver started at http://127.29.145.1:40667/ using document root <none> and password file <none>
I20260811 11:10:42.888765 5588 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:42.888850 5588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:42.890585 5588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:42.891295 5713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.891505 5588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:42.891597 5588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
format_stamp: "Formatted at 2026-08-11 11:10:41 on dist-test-slave-69rg"
server_key: "2a6580543937bcfc454927fd24851c8c"
server_key_iv: "0b46a14aa294ce7246f49c817c2caaee"
server_key_version: "encryptionkey@0"
I20260811 11:10:42.892014 5588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:42.917981 5588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:42.920754 5588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:42.920920 5588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:42.921206 5588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:42.921568 5588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:42.921604 5588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.921627 5588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:42.921664 5588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:42.927775 5588 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:46827
I20260811 11:10:42.927877 5845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:46827 every 8 connection(s)
I20260811 11:10:42.928220 5588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:42.928471 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 5588
I20260811 11:10:42.928638 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4955
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
Aug 11 11:10:42 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446642, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:42.948226 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.937428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:49139 (local address 127.29.145.61:37055)
0811 11:10:42.937475 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.937477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.937594 (+ 117us) server_negotiation.cc:407] Connection header received
0811 11:10:42.937689 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.937690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.937702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.937732 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.938203 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.938923 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.940908 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.941123 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.941155 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.944668 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.944670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.944670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.944687 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.946957 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.947389 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.947390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.947391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.947451 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.947650 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.947651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.947651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.947809 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.947856 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.948055 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.948138 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:10:42.948311 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.929285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:54735 (local address 127.29.145.60:44557)
0811 11:10:42.931236 (+ 1951us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.931239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.936656 (+ 5417us) server_negotiation.cc:407] Connection header received
0811 11:10:42.936729 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.936730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.936748 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.937811 (+ 1063us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.937883 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.938966 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.940141 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.940457 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.940522 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.943968 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.943969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.943970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.943976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.946114 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.946672 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.946674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.946674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.946740 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.947033 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.947034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.947034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.947177 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.947258 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.948110 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.948208 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1921,"thread_start_us":688,"threads_started":1}
I20260811 11:10:42.948025 5853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.928966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.29.145.1:54735)
0811 11:10:42.936569 (+ 7603us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.936581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.936669 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.936940 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.936945 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.937373 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.937713 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.937719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.939090 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.939091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.940040 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.940043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.940590 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.940591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.940736 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.941582 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.941591 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.943833 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.946228 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.946235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.946238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.946580 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.946818 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.946819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.946819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.946954 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.947286 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.947293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.947461 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.947573 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.947754 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7577,"mutex_wait_us":23,"thread_start_us":93,"threads_started":1}
I20260811 11:10:42.949493 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:35361
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:45883
--webserver_port=34979
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:10:42.949868 5846 heartbeater.cc:347] Connected to a master server at 127.29.145.60:44557
I20260811 11:10:42.949970 5846 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:42.950169 5846 heartbeater.cc:511] Master 127.29.145.60:44557 requested a full tablet report, sending...
I20260811 11:10:42.950685 5631 ts_manager.cc:194] Registered new tserver with Master: 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:42.951424 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.928964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:42839)
0811 11:10:42.930082 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.930145 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.932413 (+ 2268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.938211 (+ 5798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.938212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.938431 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.938516 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.938519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.943930 (+ 5411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.943931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.944822 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.944825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.945445 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.945446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.945522 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.945966 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.945971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.947575 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.949528 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.949529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.949529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.949748 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.950500 (+ 752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.950501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.950501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.950565 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.950860 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.950861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.950904 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.950965 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.951077 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":100,"threads_started":1}
I20260811 11:10:42.949033 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.937093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.29.145.1:49139)
0811 11:10:42.937531 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:42.937541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:42.937617 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:42.937799 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:42.937800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:42.938000 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:42.938056 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.938058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.939077 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.939078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.940804 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:42.940806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.941228 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.941228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:42.941273 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.941795 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:42.941799 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:42.944574 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:42.947039 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.947040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.947040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.947232 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.947524 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:42.947525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:42.947525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:42.947580 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:42.947906 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:42.947907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:42.947944 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 11:10:42.948777 (+ 833us) client_negotiation.cc:240] Negotiation successful
0811 11:10:42.948895 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":46,"thread_start_us":222,"threads_started":1}
I20260811 11:10:42.951829 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:42.937925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:42839 (local address 127.29.145.62:45883)
0811 11:10:42.937988 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:42.937990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:42.938002 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:42.938034 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:42.938035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:42.938049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:42.938145 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:42.942661 (+ 4516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.943485 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.944943 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:42.945325 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:42.945387 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:42.947669 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:42.947670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:42.947671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:42.947678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:42.949446 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.950345 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.950347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.950347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.950413 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:42.950630 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:42.950631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:42.950631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:42.950771 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:42.950823 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:42.951622 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 11:10:42.951720 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:10:42.952116 5849 heartbeater.cc:347] Connected to a master server at 127.29.145.61:37055
I20260811 11:10:42.952179 5849 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:42.952260 5849 heartbeater.cc:511] Master 127.29.145.61:37055 requested a full tablet report, sending...
I20260811 11:10:42.952342 5848 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:42.952378 5848 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:42.952430 5848 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:42.952634 5418 ts_manager.cc:194] Registered new tserver with Master: 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:42.952709 4724 ts_manager.cc:194] Re-registered known tserver with Master: 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:42.953212 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:42839
W20260811 11:10:43.035497 5857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:43.035704 5857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:43.035727 5857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:43.036803 5857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:43.036841 5857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:43.036868 5857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:43.038465 5857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:35361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=34979
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:43.038666 5857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:43.038851 5857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:43.044246 5862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:43.044296 5865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:43.044250 5863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:43.044479 5857 server_base.cc:1034] running on GCE node
I20260811 11:10:43.044700 5857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:43.044914 5857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:43.046069 5857 hybrid_clock.cc:648] HybridClock initialized: now 1786446643046053 us; error 34 us; skew 500 ppm
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:43.048862 5857 init.cc:377] Logged in from keytab as oryx/127.29.145.2@KRBTEST.COM (short username oryx)
I20260811 11:10:43.049351 5857 webserver.cc:460] Webserver started at http://127.29.145.2:34979/ using document root <none> and password file <none>
I20260811 11:10:43.049598 5857 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:43.049687 5857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:43.051323 5857 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:43.052034 5872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.052297 5857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:43.052441 5857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "c48342c3ef348510203e77e3e802f602"
server_key_iv: "fbeb7d6cc96157fd1e4891f4ea68729f"
server_key_version: "encryptionkey@0"
I20260811 11:10:43.052840 5857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:43.074286 5857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:43.076918 5857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:43.077087 5857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:43.077369 5857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:43.077708 5857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:43.077764 5857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.077800 5857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:43.077870 5857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.083873 5857 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35361
I20260811 11:10:43.083930 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35361 every 8 connection(s)
I20260811 11:10:43.084316 5857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:43.086385 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 5857
I20260811 11:10:43.086567 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 5110
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:43.097002 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:35027 (local address 127.29.145.60:44557)
0811 11:10:43.085157 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.085160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.085415 (+ 255us) server_negotiation.cc:407] Connection header received
0811 11:10:43.086150 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.086152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.086170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.086224 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.088191 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.088779 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.089964 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.090203 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.090241 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.093640 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.093641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.093641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.093649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.095385 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.095840 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.095842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.095842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.095892 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.096173 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.096174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.096174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.096322 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.096361 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.096757 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.096870 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:10:43.097628 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.29.145.2:35027)
0811 11:10:43.085269 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.085313 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.086029 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.086292 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.086296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.088030 (+ 1734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.088092 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.088095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.088883 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.088883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.089871 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.089873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.090321 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.090322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.090441 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.090990 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.091001 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.093533 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.095471 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.095478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.095480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.095766 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.095990 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.095991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.095991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.096109 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.096420 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.096426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.096580 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.097166 (+ 586us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.097361 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":31,"thread_start_us":101,"threads_started":1}
I20260811 11:10:43.097853 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:59117 (local address 127.29.145.62:45883)
0811 11:10:43.085257 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.085260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.085448 (+ 188us) server_negotiation.cc:407] Connection header received
0811 11:10:43.086291 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.086293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.086305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.086365 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.087447 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.088457 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.090271 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.090678 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.090730 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.093739 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.093741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.093741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.093750 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.095728 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.096268 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.096270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.096270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.096343 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.096653 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.096654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.096655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.096828 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.096905 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.097568 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.097677 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 11:10:43.098824 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:59117)
0811 11:10:43.085367 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.085377 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.086044 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.086400 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.086402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.086959 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.087310 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.087318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.088614 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.088615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.090145 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.090149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.090826 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.090827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.090921 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.091335 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.091341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.093611 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.095850 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.095852 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.095852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.096143 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.096461 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.096462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.096463 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.096537 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.096999 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.097000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.097395 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.098505 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.098652 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":52,"thread_start_us":102,"threads_started":1}
I20260811 11:10:43.102085 6001 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:43.102224 6001 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.102299 6000 heartbeater.cc:347] Connected to a master server at 127.29.145.60:44557
I20260811 11:10:43.102368 6000 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.102460 6001 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:43.102471 6000 heartbeater.cc:511] Master 127.29.145.60:44557 requested a full tablet report, sending...
I20260811 11:10:43.103094 5631 ts_manager.cc:194] Registered new tserver with Master: 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2:35361)
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
I20260811 11:10:43.103622 4724 ts_manager.cc:194] Re-registered known tserver with Master: 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2:35361)
I20260811 11:10:43.104177 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:59117
I20260811 11:10:43.106734 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:36813 (local address 127.29.145.61:37055)
0811 11:10:43.085721 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.085724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.085734 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:43.086429 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.086431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.086444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.086494 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.098488 (+ 11994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.099307 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.100651 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.100911 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.100974 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.103813 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.103815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.103815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.103824 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.105514 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.105910 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.105912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.105912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.105982 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.106198 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.106199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.106199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.106315 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.106384 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.106548 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.106637 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:10:43.106753 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.085414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.29.145.2:36813)
0811 11:10:43.085524 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.085537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.086044 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.086545 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.086545 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.098328 (+ 11783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.098394 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.098396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.099405 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.099407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.100549 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.100553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.100995 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.100996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.101084 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.101613 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.101619 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.103733 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.105603 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.105605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.105605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.105784 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.106055 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.106056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.106057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.106125 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.106389 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.106390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.106431 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.106507 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.106630 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
I20260811 11:10:43.107344 6002 heartbeater.cc:347] Connected to a master server at 127.29.145.61:37055
I20260811 11:10:43.107405 6002 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.107492 6002 heartbeater.cc:511] Master 127.29.145.61:37055 requested a full tablet report, sending...
I20260811 11:10:43.107827 5418 ts_manager.cc:194] Registered new tserver with Master: 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2:35361)
I20260811 11:10:43.107836 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:33451
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:45883
--webserver_port=40743
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:43.199704 6008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:43.199905 6008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:43.199929 6008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:43.201076 6008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:43.201125 6008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:43.201154 6008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:43.202757 6008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:33451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=40743
--tserver_master_addrs=127.29.145.62:45883,127.29.145.61:37055,127.29.145.60:44557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:43.203013 6008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:43.203266 6008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:43.208981 6018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:43.208997 6016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:43.209010 6015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:43.209369 6008 server_base.cc:1034] running on GCE node
I20260811 11:10:43.209587 6008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:43.209889 6008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:43.211069 6008 hybrid_clock.cc:648] HybridClock initialized: now 1786446643211033 us; error 52 us; skew 500 ppm
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:43.213914 6008 init.cc:377] Logged in from keytab as oryx/127.29.145.3@KRBTEST.COM (short username oryx)
I20260811 11:10:43.214413 6008 webserver.cc:460] Webserver started at http://127.29.145.3:40743/ using document root <none> and password file <none>
I20260811 11:10:43.214685 6008 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:43.214768 6008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:43.216216 6008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:43.216876 6025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.217068 6008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:43.217186 6008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
format_stamp: "Formatted at 2026-08-11 11:10:42 on dist-test-slave-69rg"
server_key: "35c08d229e4b33fb4cb980e950a41e9a"
server_key_iv: "2a873db3f0669a35b38bace38e154f2e"
server_key_version: "encryptionkey@0"
I20260811 11:10:43.217573 6008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:43.242228 6008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:43.244748 6008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:43.244874 6008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:43.245199 6008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:43.245483 6008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:43.245513 6008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.245539 6008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:43.245558 6008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:43.251602 6008 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:33451
I20260811 11:10:43.251690 6159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:33451 every 8 connection(s)
I20260811 11:10:43.252104 6008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:43.254347 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 6008
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.60@KRBTEST.COM
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.61@KRBTEST.COM
Aug 11 11:10:43 dist-test-slave-69rg krb5kdc[4671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446643, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:43.265512 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.252951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.29.145.3:48687)
0811 11:10:43.253184 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.253229 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.253860 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.254120 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.254124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.254927 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.255207 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.255213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.257731 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.257732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.258542 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.258545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.259039 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.259040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.259106 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.259467 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.259471 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.261948 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.263811 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.263819 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.263821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.264135 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.264374 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.264375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.264375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.264497 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.264765 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.264772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.264949 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.265061 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.265241 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":14,"thread_start_us":122,"threads_started":1}
I20260811 11:10:43.265895 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.252942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:48687 (local address 127.29.145.60:44557)
0811 11:10:43.252999 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.253002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.253331 (+ 329us) server_negotiation.cc:407] Connection header received
0811 11:10:43.253974 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.253975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.253988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.254042 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.256950 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.257650 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.258630 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.258916 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.258985 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.262060 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.262061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.262061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.262070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.263727 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.264231 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.264233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.264233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.264297 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.264576 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.264577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.264577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.264697 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.264774 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.265670 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.265761 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":25}
I20260811 11:10:43.266613 6160 heartbeater.cc:347] Connected to a master server at 127.29.145.60:44557
I20260811 11:10:43.266716 6160 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.266921 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.253283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.29.145.3:51427)
0811 11:10:43.253429 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.253438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.253865 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.254386 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.254388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.254941 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.255207 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.255213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.256929 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.256931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.257872 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.257875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.258365 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.258366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.258432 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.258924 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.258935 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.262585 (+ 3650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.265631 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.265632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.265632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.265881 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.266135 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.266136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.266136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.266198 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.266506 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.266508 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.266557 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.266632 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.266822 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":11,"thread_start_us":86,"threads_started":1}
I20260811 11:10:43.266920 6160 heartbeater.cc:511] Master 127.29.145.60:44557 requested a full tablet report, sending...
I20260811 11:10:43.266989 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.253370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:51427 (local address 127.29.145.61:37055)
0811 11:10:43.253419 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.253421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.253512 (+ 91us) server_negotiation.cc:407] Connection header received
0811 11:10:43.254276 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.254278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.254290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.254349 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.256128 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.256853 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.257957 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.258259 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.258316 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.262681 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.262682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.262682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.262690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.265533 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.265996 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.265997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.265998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.266059 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.266269 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.266271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.266271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.266411 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.266475 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.266796 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.266873 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:43.267390 6162 heartbeater.cc:347] Connected to a master server at 127.29.145.61:37055
I20260811 11:10:43.267438 6162 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.267513 6162 heartbeater.cc:511] Master 127.29.145.61:37055 requested a full tablet report, sending...
I20260811 11:10:43.267565 5631 ts_manager.cc:194] Registered new tserver with Master: 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:43.267809 5418 ts_manager.cc:194] Registered new tserver with Master: 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:43.268573 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.252988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:36233)
0811 11:10:43.253484 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.253502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.253869 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.254401 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.254403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.254888 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:43.255207 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.255215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.256122 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.256124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.257443 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.257446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.258018 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.258019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.258184 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.259275 (+ 1091us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:43.259279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:43.263702 (+ 4423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:43.266549 (+ 2847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.266550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.266550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.267652 (+ 1102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.267856 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:43.267857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:43.267858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:43.267921 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:43.268250 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:43.268251 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:43.268289 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.268371 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.268473 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":32,"thread_start_us":104,"threads_started":1}
I20260811 11:10:43.268600 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.253109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:36233 (local address 127.29.145.62:45883)
0811 11:10:43.253215 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.253218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.253569 (+ 351us) server_negotiation.cc:407] Connection header received
0811 11:10:43.254148 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.254149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.254162 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.254219 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.255338 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.256026 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.257542 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.257885 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.257928 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.263841 (+ 5913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.263842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.263842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.263850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.266469 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.267717 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.267718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.267719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.267775 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.268017 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.268018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.268019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.268172 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.268248 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.268403 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.268505 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 11:10:43.269244 6161 heartbeater.cc:347] Connected to a master server at 127.29.145.62:45883
I20260811 11:10:43.269302 6161 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:43.269376 6161 heartbeater.cc:511] Master 127.29.145.62:45883 requested a full tablet report, sending...
I20260811 11:10:43.269621 4724 ts_manager.cc:194] Re-registered known tserver with Master: 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:43.269708 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.256247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.29.145.61:37055)
0811 11:10:43.256512 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.256515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.256754 (+ 239us) server_negotiation.cc:407] Connection header received
0811 11:10:43.256890 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.256891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.256902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.256945 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.257649 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.258897 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.260188 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.260518 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.260588 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.262080 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.262082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.262082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.262088 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.268416 (+ 6328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.268827 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.268829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.268829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.268905 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.269117 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.269117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.269118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.269235 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.269354 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.269518 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.269603 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":65,"threads_started":1}
I20260811 11:10:43.270146 4724 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:36233
I20260811 11:10:43.270884 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.255651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.29.145.60:44557)
0811 11:10:43.255900 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.255902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.255910 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:43.255935 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.255936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.255949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.258675 (+ 2726us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.258717 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.259459 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.260454 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.260804 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.261139 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.262254 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.262255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.262255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.262262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.269545 (+ 7283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.270020 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.270021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.270021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.270075 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.270304 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.270305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.270305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.270452 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.270549 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.270753 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.270825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":227,"thread_start_us":77,"threads_started":1}
I20260811 11:10:43.272760 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.255925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.29.145.62:45883)
0811 11:10:43.260232 (+ 4307us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.260234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.260241 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:10:43.260260 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.260261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.260273 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.261530 (+ 1257us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.261584 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.263630 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.265131 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.265462 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.265781 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.267013 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.267015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.267015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.267022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.271621 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.271917 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.271918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.271918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.271969 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.272151 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.272151 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.272152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.272237 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.272462 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.272510 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.272655 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4274,"thread_start_us":101,"threads_started":1}
I20260811 11:10:43.274307 4724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:43.276127 4724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:43.284221 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.29.145.3:33451)
0811 11:10:43.280704 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.280710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.280721 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:43.280764 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.280766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.280783 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.280879 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.281089 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.282021 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283075 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.283907 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283966 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.284049 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.284105 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.284144 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":73,"threads_started":1}
I20260811 11:10:43.284210 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:33451 (local address 127.0.0.1:35148)
0811 11:10:43.280213 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.280226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.280274 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.280958 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.280959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.280961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.281006 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.281012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.282102 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.282103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.282977 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.282979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.283972 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.284031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.284035 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.284087 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.284137 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 11:10:43.284989 6079 tablet_service.cc:1467] Processing CreateTablet for tablet 3c163b967783468abb11ee3a06497793 (DEFAULT_TABLE table=test-table [id=063a20af499a4fe590fb0df6323c3498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:43.285429 6079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c163b967783468abb11ee3a06497793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:43.285423 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35361 (local address 127.0.0.1:47744)
0811 11:10:43.280409 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.280419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.280465 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.281379 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.281380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.281383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.281422 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.281424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.283019 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.283861 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.283866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.284873 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.284874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.285233 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.285238 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.285297 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.285351 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":132,"threads_started":1}
I20260811 11:10:43.285612 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46827 (local address 127.0.0.1:57000)
0811 11:10:43.280557 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.280565 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.280632 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.281094 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.281096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.281098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.281144 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.281146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.282668 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.282669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.283813 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.283817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.285363 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.285364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.285422 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.285426 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.285483 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.285527 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":13,"thread_start_us":77,"threads_started":1}
I20260811 11:10:43.285766 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.29.145.2:35361)
0811 11:10:43.281210 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.281215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.281225 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:43.281262 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.281264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.281281 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.281363 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.281496 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.282955 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283960 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.284707 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.284773 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.284837 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.285613 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.285666 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":101,"threads_started":1}
I20260811 11:10:43.286314 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.280631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.29.145.1:46827)
0811 11:10:43.280924 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.280928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.280937 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:43.280982 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.280983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.280995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.281081 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.281227 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.282519 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.283919 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.285181 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.285245 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.285329 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.286179 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.286232 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":97,"threads_started":1}
I20260811 11:10:43.286489 5925 tablet_service.cc:1467] Processing CreateTablet for tablet 3c163b967783468abb11ee3a06497793 (DEFAULT_TABLE table=test-table [id=063a20af499a4fe590fb0df6323c3498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:43.286836 5925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c163b967783468abb11ee3a06497793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:43.286914 5768 tablet_service.cc:1467] Processing CreateTablet for tablet 3c163b967783468abb11ee3a06497793 (DEFAULT_TABLE table=test-table [id=063a20af499a4fe590fb0df6323c3498]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:43.287228 5768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c163b967783468abb11ee3a06497793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:43.290053 6188 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Bootstrap starting.
I20260811 11:10:43.290715 6188 tablet_bootstrap.cc:654] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:43.291077 6188 log.cc:824] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:43.291194 6189 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Bootstrap starting.
I20260811 11:10:43.291317 6190 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Bootstrap starting.
I20260811 11:10:43.291982 6190 tablet_bootstrap.cc:654] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:43.291981 6189 tablet_bootstrap.cc:654] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:43.292327 6189 log.cc:824] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:43.292363 6190 log.cc:824] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:43.293195 6190 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: No bootstrap required, opened a new log
I20260811 11:10:43.293288 6190 ts_tablet_manager.cc:1397] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:43.294113 6189 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: No bootstrap required, opened a new log
I20260811 11:10:43.294190 6189 ts_tablet_manager.cc:1397] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:10:43.296314 6190 raft_consensus.cc:348] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.296764 6190 raft_consensus.cc:374] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:43.296854 6190 raft_consensus.cc:729] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3367ff3c204053a1bdbc6f1a8acfa0, State: Initialized, Role: FOLLOWER
I20260811 11:10:43.296993 6190 consensus_queue.cc:260] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.297103 6189 raft_consensus.cc:348] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.297286 6189 raft_consensus.cc:374] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:43.297322 6189 raft_consensus.cc:729] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285be6dd4a3a46eda6cfbacc6ffdcc78, State: Initialized, Role: FOLLOWER
I20260811 11:10:43.297363 6190 ts_tablet_manager.cc:1428] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 11:10:43.297459 6189 consensus_queue.cc:260] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.297626 6001 heartbeater.cc:503] Master 127.29.145.62:45883 was elected leader, sending a full tablet report...
I20260811 11:10:43.297801 6189 ts_tablet_manager.cc:1428] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 11:10:43.298820 5848 heartbeater.cc:503] Master 127.29.145.62:45883 was elected leader, sending a full tablet report...
I20260811 11:10:43.299961 6188 tablet_bootstrap.cc:492] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: No bootstrap required, opened a new log
I20260811 11:10:43.300055 6188 ts_tablet_manager.cc:1397] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260811 11:10:43.302380 6188 raft_consensus.cc:348] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.302520 6188 raft_consensus.cc:374] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:43.302541 6188 raft_consensus.cc:729] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c62b03fbeb4f6e9e3d8c7ce84957ed, State: Initialized, Role: FOLLOWER
I20260811 11:10:43.302670 6188 consensus_queue.cc:260] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.302960 6188 ts_tablet_manager.cc:1428] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:43.303064 6161 heartbeater.cc:503] Master 127.29.145.62:45883 was elected leader, sending a full tablet report...
I20260811 11:10:43.320087 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.317390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45040)
0811 11:10:43.317460 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.317475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.317541 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.317697 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.317698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.317699 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.317750 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.317751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.318572 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.318573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.319316 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.319319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.319884 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.319884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.319935 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.319939 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.319983 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.320025 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 11:10:43.320156 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.317465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.29.145.60:44557)
0811 11:10:43.317545 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.317546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.317576 (+ 30us) server_negotiation.cc:407] Connection header received
0811 11:10:43.317600 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.317601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.317626 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.317677 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.317811 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.318435 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.319397 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.319806 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.319866 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.319926 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.319997 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.320027 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:43.320573 5646 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260811 11:10:43.321945 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcdecc366cc8437a9b09550699839976. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:43.322045 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: LEARNER
W20260811 11:10:43.335089 6003 tablet.cc:2369] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:43.345412 6194 raft_consensus.cc:482] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:43.345579 6194 raft_consensus.cc:504] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.346066 6194 leader_election.cc:288] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451), 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:43.349990 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.346370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:52217 (local address 127.29.145.1:46827)
0811 11:10:43.346417 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.346419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.346728 (+ 309us) server_negotiation.cc:407] Connection header received
0811 11:10:43.346761 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.346762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.346772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.346825 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.347031 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.347698 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.348962 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.349537 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.349616 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.349626 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.349869 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.349920 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:10:43.349990 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.346336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:52445 (local address 127.29.145.3:33451)
0811 11:10:43.346398 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.346401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.346471 (+ 70us) server_negotiation.cc:407] Connection header received
0811 11:10:43.346555 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.346557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.346568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.346624 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:43.346802 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.347525 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.348956 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.349535 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.349625 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.349633 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.349875 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.349920 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 11:10:43.349987 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.346273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:33451 (local address 127.29.145.2:52445)
0811 11:10:43.346423 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.346434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.346495 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.346647 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.346648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.346651 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.346736 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.346738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.347628 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.347629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.348849 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.348852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.349645 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.349646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.349739 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.349744 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.349832 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.349884 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":12}
I20260811 11:10:43.349997 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.346280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46827 (local address 127.29.145.2:52217)
0811 11:10:43.346646 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:43.346657 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:43.346712 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:43.346877 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:43.346878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:43.346880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:43.346953 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.346955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.347775 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.347776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.348849 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:43.348852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.349645 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.349646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:43.349739 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.349744 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:43.349838 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 11:10:43.349884 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":278}
I20260811 11:10:43.350437 5789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c163b967783468abb11ee3a06497793" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" is_pre_election: true
I20260811 11:10:43.350502 6103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c163b967783468abb11ee3a06497793" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" is_pre_election: true
I20260811 11:10:43.350620 6103 raft_consensus.cc:2454] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 0.
I20260811 11:10:43.350621 5789 raft_consensus.cc:2454] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 0.
I20260811 11:10:43.350862 5876 leader_election.cc:302] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33c62b03fbeb4f6e9e3d8c7ce84957ed, 5f3367ff3c204053a1bdbc6f1a8acfa0; no voters:
I20260811 11:10:43.350992 6194 raft_consensus.cc:2781] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:43.351049 6194 raft_consensus.cc:482] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:43.351073 6194 raft_consensus.cc:3037] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:43.351809 6194 raft_consensus.cc:504] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.351965 6194 leader_election.cc:288] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 election: Requested vote from peers 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451), 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827)
I20260811 11:10:43.352110 6103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c163b967783468abb11ee3a06497793" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
I20260811 11:10:43.352111 5789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c163b967783468abb11ee3a06497793" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
I20260811 11:10:43.352207 6103 raft_consensus.cc:3037] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:43.352252 5789 raft_consensus.cc:3037] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:43.352983 5789 raft_consensus.cc:2454] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 1.
I20260811 11:10:43.353047 6103 raft_consensus.cc:2454] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 1.
I20260811 11:10:43.353240 5876 leader_election.cc:302] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33c62b03fbeb4f6e9e3d8c7ce84957ed, 5f3367ff3c204053a1bdbc6f1a8acfa0; no voters:
I20260811 11:10:43.353353 6194 raft_consensus.cc:2781] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:43.353533 6194 raft_consensus.cc:686] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 LEADER]: Becoming Leader. State: Replica: 5f3367ff3c204053a1bdbc6f1a8acfa0, State: Running, Role: LEADER
I20260811 11:10:43.353645 6194 consensus_queue.cc:237] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } }
I20260811 11:10:43.354635 4724 catalog_manager.cc:5495] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:43.355331 5369 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: NON_VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: true } } }
I20260811 11:10:43.355429 5369 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:43.356060 5369 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: attempting to promote NON_VOTER e65b9e270550410b808b9cda6e3145b6 to VOTER
W20260811 11:10:43.356184 5591 consensus_queue.cc:1159] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1f0dcbba5423443b8c6c887bec4c1218" responder_term: 1 status { last_received { term: 1 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 1 index: 10 } }
I20260811 11:10:43.356328 5369 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:43.356731 5428 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260811 11:10:43.356729 5646 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 LEARNER]: Refusing update from remote peer dcdecc366cc8437a9b09550699839976: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260811 11:10:43.356913 5368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 11:10:43.356989 5591 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260811 11:10:43.357676 5368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e65b9e270550410b808b9cda6e3145b6 (127.29.145.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.357800 5428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e65b9e270550410b808b9cda6e3145b6 (127.29.145.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.358537 5368 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.358646 5368 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:43.358610 5369 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.358698 5369 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:43.358704 5368 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.358848 5368 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:43.358788 6206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.358961 6206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:43.358891 5646 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, e65b9e270550410b808b9cda6e3145b6 (127.29.145.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.360065 5682 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dcdecc366cc8437a9b09550699839976" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:43.360152 5682 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:43.363216 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.360944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47750 (local address 127.29.145.2:35361)
0811 11:10:43.360980 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.360982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.360989 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:10:43.361048 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.361049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.361065 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.361124 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:43.361205 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.361665 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.362521 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.362712 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.362770 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.362940 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:43.363033 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:43.363094 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.363149 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.363166 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:10:43.364950 5789 raft_consensus.cc:1261] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Refusing update from remote peer 5f3367ff3c204053a1bdbc6f1a8acfa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:43.365295 6103 raft_consensus.cc:1261] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Refusing update from remote peer 5f3367ff3c204053a1bdbc6f1a8acfa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:43.365295 6194 consensus_queue.cc:1035] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:43.365662 6194 consensus_queue.cc:1035] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:43.368362 6210 mvcc.cc:205] Tried to move back new op lower bound from 7317285451220193280 to 7317285451177664512. Current Snapshot: MvccSnapshot[applied={T|T < 7317285451220193280}]
I20260811 11:10:43.369953 6209 mvcc.cc:205] Tried to move back new op lower bound from 7317285451220193280 to 7317285451177664512. Current Snapshot: MvccSnapshot[applied={T|T < 7317285451220193280 or (T in {7317285451220193280})}]
I20260811 11:10:43.372030 4722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:43.372128 4722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:43.373550 4722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dcdecc366cc8437a9b09550699839976: Sending DeleteTablet for 3 replicas of tablet 3c163b967783468abb11ee3a06497793
I20260811 11:10:43.373898 6079 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c163b967783468abb11ee3a06497793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:43 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:35148
I20260811 11:10:43.373895 5925 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c163b967783468abb11ee3a06497793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:43 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:47744
I20260811 11:10:43.374178 6220 tablet_replica.cc:330] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: stopping tablet replica
I20260811 11:10:43.374177 5768 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c163b967783468abb11ee3a06497793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:43 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:57000
I20260811 11:10:43.374284 6220 raft_consensus.cc:2229] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:43.374352 6220 raft_consensus.cc:2258] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:43.374804 6219 tablet_replica.cc:330] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: stopping tablet replica
I20260811 11:10:43.374917 6219 raft_consensus.cc:2229] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:43.375013 6219 raft_consensus.cc:2258] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:43.375177 6222 tablet_replica.cc:330] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: stopping tablet replica
I20260811 11:10:43.375299 6222 raft_consensus.cc:2229] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:43.375526 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 4687
I20260811 11:10:43.375547 6222 raft_consensus.cc:2258] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:43.375844 6219 ts_tablet_manager.cc:1905] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:43.379153 6222 ts_tablet_manager.cc:1905] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:43.379423 6219 ts_tablet_manager.cc:1918] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:43.379570 6219 log.cc:1196] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/3c163b967783468abb11ee3a06497793
I20260811 11:10:43.380249 6219 ts_tablet_manager.cc:1939] T 3c163b967783468abb11ee3a06497793 P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting consensus metadata
I20260811 11:10:43.381093 6222 ts_tablet_manager.cc:1918] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:43.381178 6222 log.cc:1196] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/3c163b967783468abb11ee3a06497793
I20260811 11:10:43.381479 6222 ts_tablet_manager.cc:1939] T 3c163b967783468abb11ee3a06497793 P 285be6dd4a3a46eda6cfbacc6ffdcc78: Deleting consensus metadata
I20260811 11:10:43.383606 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.375076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.29.145.60:44557)
0811 11:10:43.375122 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.375124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.375132 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:43.375154 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.375155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.375164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.375218 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.376341 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.377012 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.378036 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.378319 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.378358 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.380462 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.380463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.380464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.380472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.382332 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.382699 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.382701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.382701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.382761 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.383000 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.383001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.383001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.383132 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.383237 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.383411 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.383499 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 11:10:43.392231 5714 connection.cc:535] server connection from 127.0.0.1:57000 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 11:10:43.392262 5875 connection.cc:535] server connection from 127.0.0.1:47744 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:10:43.394107 6220 ts_tablet_manager.cc:1905] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:43.395973 6220 ts_tablet_manager.cc:1918] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:43.396054 6220 log.cc:1196] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/3c163b967783468abb11ee3a06497793
I20260811 11:10:43.396344 6220 ts_tablet_manager.cc:1939] T 3c163b967783468abb11ee3a06497793 P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting consensus metadata
W20260811 11:10:43.396740 6032 connection.cc:441] server connection from 127.0.0.1:35148 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35148 (request call id 1) could send its response
I20260811 11:10:43.402833 6176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.394263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.29.145.61:37055)
0811 11:10:43.394303 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.394306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.394801 (+ 495us) server_negotiation.cc:407] Connection header received
0811 11:10:43.394897 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.394898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.394913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.394946 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.395774 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.396582 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.397837 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.398158 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.398249 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.399655 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.399656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.399656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.399663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.401446 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.401944 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.401945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.401945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.402006 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.402211 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.402213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.402213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.402378 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.402481 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.402622 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.402719 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 11:10:43.403177 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:43.394208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.29.145.60:44557)
0811 11:10:43.394290 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:43.394292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:43.394577 (+ 285us) server_negotiation.cc:407] Connection header received
0811 11:10:43.394652 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:43.394653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:43.394666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:43.394720 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:43.395885 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.396618 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.397770 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:43.398154 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:43.398235 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:43.399908 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:43.399909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:43.399909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:43.399918 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:43.401581 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.402086 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.402087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.402088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.402149 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:43.402381 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:43.402382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:43.402383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:43.402511 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:43.402577 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:43.402812 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 11:10:43.403116 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:10:44.369977 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.369610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:57021)
0811 11:10:44.369766 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.369899 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
W20260811 11:10:44.370293 5848 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:45883 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.372900 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.372614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:38709)
0811 11:10:44.372750 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.372848 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260811 11:10:44.373075 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.372338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:43465)
0811 11:10:44.372490 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.372592 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
W20260811 11:10:44.373236 6001 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:45883 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
W20260811 11:10:44.373454 6161 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:45883 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.874306 6236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dcdecc366cc8437a9b09550699839976)
I20260811 11:10:44.874511 6236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:44.874871 6236 leader_election.cc:288] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557)
I20260811 11:10:44.875309 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.875025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49042)
0811 11:10:44.875171 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.875240 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260811 11:10:44.875466 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dcdecc366cc8437a9b09550699839976)
W20260811 11:10:44.875497 5402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.875620 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:44.875934 6240 leader_election.cc:288] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61:37055)
I20260811 11:10:44.875962 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.875843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49044)
0811 11:10:44.875889 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.875916 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 11:10:44.876067 5402 leader_election.cc:334] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.876302 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.876013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49058)
0811 11:10:44.876151 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.876259 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
W20260811 11:10:44.876433 5618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.876986 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.876869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49060)
0811 11:10:44.876911 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.876949 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 11:10:44.877148 5618 leader_election.cc:334] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.880105 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.875057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:44557 (local address 127.0.0.1:45064)
0811 11:10:44.875238 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.875251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:44.875332 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:44.875892 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:44.875894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:44.875898 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:44.876113 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:44.876118 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.877557 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.877558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:44.878901 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:44.878905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.879792 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:44.879869 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:44.879876 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:10:44.879944 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:10:44.879999 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260811 11:10:44.880185 6241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.875228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.29.145.60:44557)
0811 11:10:44.875362 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:44.875366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:44.875717 (+ 351us) server_negotiation.cc:407] Connection header received
0811 11:10:44.875760 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:44.875761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:44.875778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:44.875854 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:44.876430 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.877482 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879030 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.879684 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879796 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:44.879815 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:44.880027 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 11:10:44.880090 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260811 11:10:44.880196 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.876238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:37055 (local address 127.0.0.1:33722)
0811 11:10:44.876375 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.876387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:44.876442 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:44.876722 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:44.876723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:44.876727 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:44.876777 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:44.876779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.877668 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.877669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:44.878902 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:44.878905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.879913 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:44.879983 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:44.879988 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:44.880048 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:10:44.880101 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260811 11:10:44.880185 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.876277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.29.145.61:37055)
0811 11:10:44.876588 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:44.876590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:44.876598 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:44.876621 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:44.876621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:44.876633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:44.876706 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:44.876847 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.877587 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879030 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:44.879806 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:44.879935 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:44.879954 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:44.880056 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 11:10:44.880119 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":266,"thread_start_us":96,"threads_started":1}
I20260811 11:10:44.880722 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f0dcbba5423443b8c6c887bec4c1218" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e65b9e270550410b808b9cda6e3145b6" is_pre_election: true
I20260811 11:10:44.880867 5646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0dcbba5423443b8c6c887bec4c1218 in term 1.
I20260811 11:10:44.880865 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e65b9e270550410b808b9cda6e3145b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1f0dcbba5423443b8c6c887bec4c1218" is_pre_election: true
I20260811 11:10:44.880987 5428 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65b9e270550410b808b9cda6e3145b6 in term 1.
I20260811 11:10:44.881070 5398 leader_election.cc:302] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f0dcbba5423443b8c6c887bec4c1218, e65b9e270550410b808b9cda6e3145b6; no voters: dcdecc366cc8437a9b09550699839976
I20260811 11:10:44.881129 5618 leader_election.cc:302] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f0dcbba5423443b8c6c887bec4c1218, e65b9e270550410b808b9cda6e3145b6; no voters: dcdecc366cc8437a9b09550699839976
I20260811 11:10:44.881171 6236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 11:10:44.881210 6236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dcdecc366cc8437a9b09550699839976)
I20260811 11:10:44.881232 6236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:10:44.881245 6240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 11:10:44.881282 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dcdecc366cc8437a9b09550699839976)
I20260811 11:10:44.881310 6240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:10:44.882441 6236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:44.882441 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:44.882589 6240 leader_election.cc:288] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 election: Requested vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), 1f0dcbba5423443b8c6c887bec4c1218 (127.29.145.61:37055)
I20260811 11:10:44.882602 6236 leader_election.cc:288] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 election: Requested vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557)
I20260811 11:10:44.882848 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.882720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49074)
0811 11:10:44.882784 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.882820 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 11:10:44.882848 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.882728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49070)
0811 11:10:44.882795 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.882818 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 11:10:44.882855 5428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e65b9e270550410b808b9cda6e3145b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1f0dcbba5423443b8c6c887bec4c1218"
I20260811 11:10:44.882979 5428 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e65b9e270550410b808b9cda6e3145b6 in current term 2: Already voted for candidate 1f0dcbba5423443b8c6c887bec4c1218 in this term.
I20260811 11:10:44.882858 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f0dcbba5423443b8c6c887bec4c1218" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e65b9e270550410b808b9cda6e3145b6"
I20260811 11:10:44.883150 5646 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f0dcbba5423443b8c6c887bec4c1218 in current term 2: Already voted for candidate e65b9e270550410b808b9cda6e3145b6 in this term.
I20260811 11:10:44.883201 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.883083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49084)
0811 11:10:44.883115 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.883174 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260811 11:10:44.883316 5402 leader_election.cc:334] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.883356 5402 leader_election.cc:302] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f0dcbba5423443b8c6c887bec4c1218; no voters: dcdecc366cc8437a9b09550699839976, e65b9e270550410b808b9cda6e3145b6
I20260811 11:10:44.883417 6236 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 11:10:44.883369 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:44.883286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49090)
0811 11:10:44.883323 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:44.883346 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 11:10:44.883518 5618 leader_election.cc:334] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:44.883553 5618 leader_election.cc:302] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e65b9e270550410b808b9cda6e3145b6; no voters: 1f0dcbba5423443b8c6c887bec4c1218, dcdecc366cc8437a9b09550699839976
I20260811 11:10:44.883634 6240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 11:10:45.371045 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:45.370693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:44939)
0811 11:10:45.370915 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:45.370969 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":123,"threads_started":1}
I20260811 11:10:45.373926 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:45.373609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:33099)
0811 11:10:45.373778 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:45.373858 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260811 11:10:45.374035 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:45.373753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:55767)
0811 11:10:45.373936 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:45.373974 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260811 11:10:46.372643 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.371946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:49075)
0811 11:10:46.372521 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.372576 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":490,"threads_started":1}
I20260811 11:10:46.374735 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.374465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:55051)
0811 11:10:46.374630 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.374678 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260811 11:10:46.374735 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.374469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:45655)
0811 11:10:46.374648 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.374687 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260811 11:10:46.474891 6253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:46.475036 6253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:46.475332 6253 leader_election.cc:288] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557)
I20260811 11:10:46.475710 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f0dcbba5423443b8c6c887bec4c1218" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e65b9e270550410b808b9cda6e3145b6" is_pre_election: true
I20260811 11:10:46.475804 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.475564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49292)
0811 11:10:46.475702 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.475754 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260811 11:10:46.475868 5646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0dcbba5423443b8c6c887bec4c1218 in term 2.
I20260811 11:10:46.476115 5398 leader_election.cc:302] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0dcbba5423443b8c6c887bec4c1218, e65b9e270550410b808b9cda6e3145b6; no voters:
I20260811 11:10:46.476241 6253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 11:10:46.476294 6253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:46.476346 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.476220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49296)
0811 11:10:46.476256 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.476287 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 11:10:46.476347 6253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 2 FOLLOWER]: Advancing to term 3
W20260811 11:10:46.476423 5402 leader_election.cc:334] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:46.477382 6253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:46.477530 6253 leader_election.cc:288] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 election: Requested vote from peers dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883), e65b9e270550410b808b9cda6e3145b6 (127.29.145.60:44557)
I20260811 11:10:46.477751 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.477617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49304)
0811 11:10:46.477672 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.477726 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:10:46.477777 5646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1f0dcbba5423443b8c6c887bec4c1218" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e65b9e270550410b808b9cda6e3145b6"
I20260811 11:10:46.477903 5646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 11:10:46.478170 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.478039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49312)
0811 11:10:46.478095 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.478147 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 11:10:46.478258 5402 leader_election.cc:334] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
I20260811 11:10:46.478837 5646 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0dcbba5423443b8c6c887bec4c1218 in term 3.
I20260811 11:10:46.479094 5398 leader_election.cc:302] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f0dcbba5423443b8c6c887bec4c1218, e65b9e270550410b808b9cda6e3145b6; no voters: dcdecc366cc8437a9b09550699839976
I20260811 11:10:46.479223 6253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 11:10:46.479409 6253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [term 3 LEADER]: Becoming Leader. State: Replica: 1f0dcbba5423443b8c6c887bec4c1218, State: Running, Role: LEADER
I20260811 11:10:46.479518 6253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } }
I20260811 11:10:46.480034 6257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f0dcbba5423443b8c6c887bec4c1218. Latest consensus state: current_term: 3 leader_uuid: "1f0dcbba5423443b8c6c887bec4c1218" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:46.480165 6257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:46.480407 6260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:46.481037 6260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:46.481141 6260 catalog_manager.cc:1259] Loaded cluster ID: fbb382d3ab0643c6a70e265c8e6b95d9
I20260811 11:10:46.481182 6260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:46.481453 6260 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:46.481572 6260 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Loaded TSK: 0
I20260811 11:10:46.481755 6260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:10:46.508119 5418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:46.510386 5418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:46.511821 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.511685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49332)
0811 11:10:46.511747 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.511786 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:10:46.511914 5646 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [term 3 FOLLOWER]: Refusing update from remote peer 1f0dcbba5423443b8c6c887bec4c1218: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 11:10:46.512231 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.512131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.0.0.1:49340)
0811 11:10:46.512170 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.512202 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 11:10:46.512283 6257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 11:10:46.512463 5402 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 -> Peer dcdecc366cc8437a9b09550699839976 (127.29.145.62:45883): Couldn't send request to peer dcdecc366cc8437a9b09550699839976. Status: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 11:10:46.513159 6262 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f0dcbba5423443b8c6c887bec4c1218. Latest consensus state: current_term: 3 leader_uuid: "1f0dcbba5423443b8c6c887bec4c1218" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:46.513315 6262 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:46.514084 6253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1f0dcbba5423443b8c6c887bec4c1218" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:46.514209 6253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:46.514559 6262 sys_catalog.cc:455] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1f0dcbba5423443b8c6c887bec4c1218" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dcdecc366cc8437a9b09550699839976" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 45883 } } peers { permanent_uuid: "1f0dcbba5423443b8c6c887bec4c1218" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 37055 } attrs { promote: false } } peers { permanent_uuid: "e65b9e270550410b808b9cda6e3145b6" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 44557 } attrs { promote: false } } }
I20260811 11:10:46.514662 6262 sys_catalog.cc:458] T 00000000000000000000000000000000 P e65b9e270550410b808b9cda6e3145b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:10:46.520784 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.516914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.29.145.1:46827)
0811 11:10:46.517070 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:46.517072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:46.517081 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:46.517141 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:46.517141 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:46.517152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:46.517302 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:46.517545 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.518406 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.519874 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.520456 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.520527 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.520545 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:46.520680 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 11:10:46.520713 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260811 11:10:46.520800 6254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.516436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46827 (local address 127.0.0.1:57006)
0811 11:10:46.516553 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.516567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:46.516632 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:46.517305 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:46.517307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:46.517309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:46.517445 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.517449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.518499 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.518501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.519785 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.519788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.520550 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.520551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.520618 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.520623 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:46.520675 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 11:10:46.520723 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":42}
I20260811 11:10:46.521131 5768 tablet_service.cc:1467] Processing CreateTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd (DEFAULT_TABLE table=test-table [id=801ec05fae2747149c03fa4ee71ac7fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:46.521224 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.516665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:33451 (local address 127.0.0.1:35154)
0811 11:10:46.516835 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.516845 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:46.516899 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:46.517958 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:46.517960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:46.517962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:46.518030 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.518032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.519131 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.519132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.520142 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.520151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.520997 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.520998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.521064 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.521070 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:10:46.521101 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 11:10:46.521142 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":101,"threads_started":1}
I20260811 11:10:46.521303 5768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e16f3f70e16440c8d2f43f3ea1255fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:46.521509 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.517243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.29.145.3:33451)
0811 11:10:46.517711 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:46.517714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:46.517726 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:46.517756 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:46.517757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:46.517769 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:46.517953 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:46.518134 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.519030 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.520277 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.520883 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.521342 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.521359 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:46.521383 (+ 24us) server_negotiation.cc:299] Negotiation successful
0811 11:10:46.521424 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":69,"threads_started":1}
I20260811 11:10:46.521503 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.517077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.29.145.2:35361)
0811 11:10:46.517252 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:46.517255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:46.517266 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:46.517303 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:46.517304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:46.517318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:46.517415 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:46.517534 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.518430 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.519790 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.520455 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.520531 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.520546 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:46.521263 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 11:10:46.521392 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260811 11:10:46.521713 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.516578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35361 (local address 127.0.0.1:47762)
0811 11:10:46.516748 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.516759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:46.516974 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:46.517425 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:46.517426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:46.517429 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:46.517466 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.517468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.518525 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.518526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.519684 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:46.519688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.521074 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.521075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:46.521131 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.521134 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:46.521181 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 11:10:46.521214 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":27,"thread_start_us":108,"threads_started":1}
I20260811 11:10:46.521715 6079 tablet_service.cc:1467] Processing CreateTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd (DEFAULT_TABLE table=test-table [id=801ec05fae2747149c03fa4ee71ac7fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:46.521870 6079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e16f3f70e16440c8d2f43f3ea1255fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:46.522019 5925 tablet_service.cc:1467] Processing CreateTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd (DEFAULT_TABLE table=test-table [id=801ec05fae2747149c03fa4ee71ac7fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:46.522156 5925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e16f3f70e16440c8d2f43f3ea1255fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:46.523329 6273 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Bootstrap starting.
I20260811 11:10:46.523849 6273 tablet_bootstrap.cc:654] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:46.523941 6275 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Bootstrap starting.
I20260811 11:10:46.524155 6274 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78: Bootstrap starting.
I20260811 11:10:46.524435 6275 tablet_bootstrap.cc:654] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:46.524510 6273 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: No bootstrap required, opened a new log
I20260811 11:10:46.524654 6273 ts_tablet_manager.cc:1397] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:46.524884 6273 raft_consensus.cc:348] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.524999 6273 raft_consensus.cc:374] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:46.525038 6273 raft_consensus.cc:729] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c62b03fbeb4f6e9e3d8c7ce84957ed, State: Initialized, Role: FOLLOWER
I20260811 11:10:46.525086 6274 tablet_bootstrap.cc:654] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:46.525105 6273 consensus_queue.cc:260] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.525391 6273 ts_tablet_manager.cc:1428] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:46.525763 6274 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78: No bootstrap required, opened a new log
I20260811 11:10:46.525867 6274 ts_tablet_manager.cc:1397] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:46.525908 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.525721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:57335)
0811 11:10:46.525775 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.525813 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 11:10:46.526072 6274 raft_consensus.cc:348] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.526154 6274 raft_consensus.cc:374] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:46.526176 6274 raft_consensus.cc:729] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285be6dd4a3a46eda6cfbacc6ffdcc78, State: Initialized, Role: FOLLOWER
I20260811 11:10:46.526229 6274 consensus_queue.cc:260] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.526340 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.526135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:58273)
0811 11:10:46.526282 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.526312 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 11:10:46.526463 6274 ts_tablet_manager.cc:1428] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:46.526790 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.526678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:53251)
0811 11:10:46.526720 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.526752 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 11:10:46.527398 5849 heartbeater.cc:503] Master 127.29.145.61:37055 was elected leader, sending a full tablet report...
I20260811 11:10:46.527518 6275 tablet_bootstrap.cc:492] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: No bootstrap required, opened a new log
I20260811 11:10:46.527609 6275 ts_tablet_manager.cc:1397] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 11:10:46.527628 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.527452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:35865)
0811 11:10:46.527566 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.527599 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 11:10:46.527760 6275 raft_consensus.cc:348] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.527887 6275 raft_consensus.cc:374] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:46.527911 6275 raft_consensus.cc:729] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3367ff3c204053a1bdbc6f1a8acfa0, State: Initialized, Role: FOLLOWER
I20260811 11:10:46.527971 6275 consensus_queue.cc:260] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.528191 6275 ts_tablet_manager.cc:1428] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:46.528610 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.528427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:35689)
0811 11:10:46.528525 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.528558 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 11:10:46.528985 6002 heartbeater.cc:503] Master 127.29.145.61:37055 was elected leader, sending a full tablet report...
I20260811 11:10:46.529052 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.528879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:35809)
0811 11:10:46.528965 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.529001 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:10:46.533735 6282 raft_consensus.cc:482] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:46.533896 6282 raft_consensus.cc:504] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.534124 6282 leader_election.cc:288] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827), 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:46.534405 5789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e16f3f70e16440c8d2f43f3ea1255fd" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" is_pre_election: true
I20260811 11:10:46.534389 6103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e16f3f70e16440c8d2f43f3ea1255fd" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" is_pre_election: true
I20260811 11:10:46.534531 5789 raft_consensus.cc:2454] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 0.
I20260811 11:10:46.534555 6103 raft_consensus.cc:2454] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 0.
I20260811 11:10:46.534725 5874 leader_election.cc:302] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285be6dd4a3a46eda6cfbacc6ffdcc78, 5f3367ff3c204053a1bdbc6f1a8acfa0; no voters:
I20260811 11:10:46.534871 6282 raft_consensus.cc:2781] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:46.534924 6282 raft_consensus.cc:482] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:46.534943 6282 raft_consensus.cc:3037] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:46.535833 6282 raft_consensus.cc:504] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.536019 6282 leader_election.cc:288] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 election: Requested vote from peers 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827), 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451)
I20260811 11:10:46.536176 5789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e16f3f70e16440c8d2f43f3ea1255fd" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78"
I20260811 11:10:46.536190 6103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e16f3f70e16440c8d2f43f3ea1255fd" candidate_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed"
I20260811 11:10:46.536274 5789 raft_consensus.cc:3037] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:46.536293 6103 raft_consensus.cc:3037] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:46.537215 5789 raft_consensus.cc:2454] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 1.
I20260811 11:10:46.537215 6103 raft_consensus.cc:2454] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3367ff3c204053a1bdbc6f1a8acfa0 in term 1.
I20260811 11:10:46.537410 5876 leader_election.cc:302] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33c62b03fbeb4f6e9e3d8c7ce84957ed, 5f3367ff3c204053a1bdbc6f1a8acfa0; no voters:
I20260811 11:10:46.537572 6282 raft_consensus.cc:2781] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:46.537781 6282 raft_consensus.cc:686] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 LEADER]: Becoming Leader. State: Replica: 5f3367ff3c204053a1bdbc6f1a8acfa0, State: Running, Role: LEADER
I20260811 11:10:46.537907 6282 consensus_queue.cc:237] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } }
I20260811 11:10:46.538290 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.538139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:57189)
0811 11:10:46.538200 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.538248 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 11:10:46.538714 5418 catalog_manager.cc:5495] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3367ff3c204053a1bdbc6f1a8acfa0" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:46.549535 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.546813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.29.145.2:35361)
0811 11:10:46.546884 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:46.546886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:46.546896 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:46.546976 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:46.546977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:46.546989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:46.547034 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:46.547191 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.547770 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.548597 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:46.548812 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:46.548871 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:46.549070 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:46.549103 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:46.549160 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:46.549288 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 11:10:46.549315 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:10:46.551025 5789 raft_consensus.cc:1261] T 6e16f3f70e16440c8d2f43f3ea1255fd P 285be6dd4a3a46eda6cfbacc6ffdcc78 [term 1 FOLLOWER]: Refusing update from remote peer 5f3367ff3c204053a1bdbc6f1a8acfa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:46.551038 6103 raft_consensus.cc:1261] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Refusing update from remote peer 5f3367ff3c204053a1bdbc6f1a8acfa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:46.551324 6282 consensus_queue.cc:1035] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285be6dd4a3a46eda6cfbacc6ffdcc78" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:46.551457 6283 consensus_queue.cc:1035] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33c62b03fbeb4f6e9e3d8c7ce84957ed" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:46.552403 6162 heartbeater.cc:503] Master 127.29.145.61:37055 was elected leader, sending a full tablet report...
I20260811 11:10:46.553138 6288 mvcc.cc:205] Tried to move back new op lower bound from 7317285464270393344 to 7317285464220012544. Current Snapshot: MvccSnapshot[applied={T|T < 7317285464270393344}]
I20260811 11:10:46.553252 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.552788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:47281)
0811 11:10:46.553155 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.553204 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
I20260811 11:10:46.553480 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.553012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:33663)
0811 11:10:46.553390 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.553439 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
I20260811 11:10:46.553543 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.553360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.2:34723)
0811 11:10:46.553466 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.553499 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 11:10:46.555406 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.555160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.3:47495)
0811 11:10:46.555337 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.555372 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 11:10:46.555418 5417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:46.555537 5417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:46.555783 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.555681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:45883 (local address 127.29.145.1:54447)
0811 11:10:46.555725 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.555750 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 11:10:46.557202 5417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f0dcbba5423443b8c6c887bec4c1218: Sending DeleteTablet for 3 replicas of tablet 6e16f3f70e16440c8d2f43f3ea1255fd
I20260811 11:10:46.557576 5768 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:46 UTC) from {username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:57006
I20260811 11:10:46.557597 6079 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:46 UTC) from {username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:35154
I20260811 11:10:46.557629 5925 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e16f3f70e16440c8d2f43f3ea1255fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:46 UTC) from {username='oryx', principal='oryx/127.29.145.61@KRBTEST.COM'} at 127.0.0.1:47762
I20260811 11:10:46.557811 6296 tablet_replica.cc:330] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: stopping tablet replica
I20260811 11:10:46.557912 6296 raft_consensus.cc:2229] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:46.558018 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 5588
I20260811 11:10:46.558048 6296 raft_consensus.cc:2258] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:46.560316 6297 tablet_replica.cc:330] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: stopping tablet replica
I20260811 11:10:46.560365 6297 raft_consensus.cc:2229] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:46.560418 6297 raft_consensus.cc:2258] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:46.562269 6297 ts_tablet_manager.cc:1905] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:46.562867 6296 ts_tablet_manager.cc:1905] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:46.564623 6297 ts_tablet_manager.cc:1918] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:46.564710 6297 log.cc:1196] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/6e16f3f70e16440c8d2f43f3ea1255fd
I20260811 11:10:46.564972 6297 ts_tablet_manager.cc:1939] T 6e16f3f70e16440c8d2f43f3ea1255fd P 5f3367ff3c204053a1bdbc6f1a8acfa0: Deleting consensus metadata
I20260811 11:10:46.565373 6296 ts_tablet_manager.cc:1918] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:46.565457 6296 log.cc:1196] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/6e16f3f70e16440c8d2f43f3ea1255fd
I20260811 11:10:46.565434 5398 catalog_manager.cc:4841] TS 5f3367ff3c204053a1bdbc6f1a8acfa0 (127.29.145.2:35361): tablet 6e16f3f70e16440c8d2f43f3ea1255fd (table test-table [id=801ec05fae2747149c03fa4ee71ac7fd]) successfully deleted
I20260811 11:10:46.565798 6296 ts_tablet_manager.cc:1939] T 6e16f3f70e16440c8d2f43f3ea1255fd P 33c62b03fbeb4f6e9e3d8c7ce84957ed: Deleting consensus metadata
I20260811 11:10:46.566315 5402 catalog_manager.cc:4841] TS 33c62b03fbeb4f6e9e3d8c7ce84957ed (127.29.145.3:33451): tablet 6e16f3f70e16440c8d2f43f3ea1255fd (table test-table [id=801ec05fae2747149c03fa4ee71ac7fd]) successfully deleted
I20260811 11:10:46.571524 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 5857
I20260811 11:10:46.571542 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:46.571386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46827 (local address 127.0.0.1:57022)
0811 11:10:46.571455 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:46.571494 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:10:46.571745 5398 catalog_manager.cc:4584] TS 285be6dd4a3a46eda6cfbacc6ffdcc78 (127.29.145.1:46827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e16f3f70e16440c8d2f43f3ea1255fd: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46827: connect: Connection refused (error 111)
I20260811 11:10:46.581524 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 6008
I20260811 11:10:46.590929 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 5266
I20260811 11:10:46.600149 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 5479
2026-08-11T11:10:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6487 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:46 dist-test-slave-69rg krb5kdc[6300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:46 dist-test-slave-69rg krb5kdc[6300](info): set up 2 sockets
Aug 11 11:10:46 dist-test-slave-69rg krb5kdc[6300](info): commencing operation
krb5kdc: starting...
W20260811 11:10:47.746665 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:47 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab.
Entry for principal oryx/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab.
I20260811 11:10:47.881935 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42489
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab
--principal=oryx/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:47.966902 6316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:47.967156 6316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:47.967196 6316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:47.968329 6316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:47.968400 6316 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:10:47.968422 6316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:47.968446 6316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:47.968490 6316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:47.968520 6316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:47.970062 6316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42489
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.62.keytab
--principal=oryx/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:47.970340 6316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:47.970575 6316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:10:47.975723 6316 server_base.cc:1034] running on GCE node
W20260811 11:10:47.975679 6327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:47.975644 6322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:47.975641 6321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:47.976078 6316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:47.976408 6316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:47.977561 6316 hybrid_clock.cc:648] HybridClock initialized: now 1786446647977536 us; error 42 us; skew 500 ppm
Aug 11 11:10:47 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446647, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:47.980146 6316 init.cc:377] Logged in from keytab as oryx/127.29.145.62@KRBTEST.COM (short username oryx)
I20260811 11:10:47.980620 6316 webserver.cc:460] Webserver started at http://127.29.145.62:43153/ using document root <none> and password file <none>
I20260811 11:10:47.980865 6316 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:47.980933 6316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:47.981087 6316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:47.982182 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "141c9727d46a41f48d238b00664bf08f"
format_stamp: "Formatted at 2026-08-11 11:10:47 on dist-test-slave-69rg"
server_key: "df1b469b59aaac22bb322933d86fc765"
server_key_iv: "5618e35eb7e1ca0813a8cc728108ca7f"
server_key_version: "encryptionkey@0"
I20260811 11:10:47.982527 6316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "141c9727d46a41f48d238b00664bf08f"
format_stamp: "Formatted at 2026-08-11 11:10:47 on dist-test-slave-69rg"
server_key: "df1b469b59aaac22bb322933d86fc765"
server_key_iv: "5618e35eb7e1ca0813a8cc728108ca7f"
server_key_version: "encryptionkey@0"
I20260811 11:10:47.983827 6316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:47.984589 6337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:47.984766 6316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:47.984875 6316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "141c9727d46a41f48d238b00664bf08f"
format_stamp: "Formatted at 2026-08-11 11:10:47 on dist-test-slave-69rg"
server_key: "df1b469b59aaac22bb322933d86fc765"
server_key_iv: "5618e35eb7e1ca0813a8cc728108ca7f"
server_key_version: "encryptionkey@0"
I20260811 11:10:47.984952 6316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:48.011682 6316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:48.013623 6316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:48.013811 6316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:48.018994 6316 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42489
I20260811 11:10:48.019042 6410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42489 every 8 connection(s)
I20260811 11:10:48.019382 6316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:48.020102 6411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:48.022541 6411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Bootstrap starting.
I20260811 11:10:48.023279 6411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:48.023583 6411 log.cc:824] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:48.024312 6411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: No bootstrap required, opened a new log
I20260811 11:10:48.026503 6411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } }
I20260811 11:10:48.026672 6411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:48.026719 6411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 141c9727d46a41f48d238b00664bf08f, State: Initialized, Role: FOLLOWER
I20260811 11:10:48.026895 6411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } }
I20260811 11:10:48.026981 6411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:48.027026 6411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:48.027077 6411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:48.027622 6411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } }
I20260811 11:10:48.027753 6411 leader_election.cc:302] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 141c9727d46a41f48d238b00664bf08f; no voters:
I20260811 11:10:48.027941 6411 leader_election.cc:288] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:48.028000 6414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:48.028352 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 6316
I20260811 11:10:48.028475 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:48.028702 30276 external_mini_cluster.cc:1428] Setting key f5316cb17380860891180319f245ed4f
I20260811 11:10:48.028226 6411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:48.028934 6414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [term 1 LEADER]: Becoming Leader. State: Replica: 141c9727d46a41f48d238b00664bf08f, State: Running, Role: LEADER
I20260811 11:10:48.029104 6414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } }
I20260811 11:10:48.029969 6417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 141c9727d46a41f48d238b00664bf08f. Latest consensus state: current_term: 1 leader_uuid: "141c9727d46a41f48d238b00664bf08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } } }
I20260811 11:10:48.030066 6417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:48.030471 6416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "141c9727d46a41f48d238b00664bf08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "141c9727d46a41f48d238b00664bf08f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42489 } } }
I20260811 11:10:48.030560 6416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:48.031088 6428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:48.031750 6428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:48.033272 6428 catalog_manager.cc:1347] Generated new cluster ID: f999194bb23743a4ac9302146bce7763
I20260811 11:10:48.033324 6428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:48.041199 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.029192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.29.145.62:42489)
0811 11:10:48.029529 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.029539 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.029570 (+ 31us) server_negotiation.cc:407] Connection header received
0811 11:10:48.029693 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.029699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.030069 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.030233 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.031121 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.032581 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.033643 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.034314 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.034424 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.037327 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.037331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.037331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.037344 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.039389 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.039848 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.039851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.039852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.039953 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.040187 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.040188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.040188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.040491 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.040640 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.040845 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.040979 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":92,"threads_started":1}
I20260811 11:10:48.043619 6428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:48.044337 6428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:48.056818 6428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Generated new TSK 0
I20260811 11:10:48.057011 6428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
I20260811 11:10:48.093847 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42489
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:48.178391 6442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:48.178653 6442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:48.178695 6442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:48.179876 6442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:48.179941 6442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:48.179979 6442 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:48.180037 6442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:48.181646 6442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:48.181962 6442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:48.182236 6442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:48.187515 6448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:48.187558 6451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:48.187745 6442 server_base.cc:1034] running on GCE node
W20260811 11:10:48.187541 6449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:48.188040 6442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:48.188256 6442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:48.189465 6442 hybrid_clock.cc:648] HybridClock initialized: now 1786446648189451 us; error 49 us; skew 500 ppm
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:48.192378 6442 init.cc:377] Logged in from keytab as oryx/127.29.145.1@KRBTEST.COM (short username oryx)
I20260811 11:10:48.192870 6442 webserver.cc:460] Webserver started at http://127.29.145.1:42825/ using document root <none> and password file <none>
I20260811 11:10:48.193140 6442 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:48.193213 6442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:48.193368 6442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:48.194517 6442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "37fadca0b1a3a57cf5450385a556b945"
server_key_iv: "d0ea5931a5b425bf189d3ba94a673406"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.194890 6442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "37fadca0b1a3a57cf5450385a556b945"
server_key_iv: "d0ea5931a5b425bf189d3ba94a673406"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.196271 6442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:48.197054 6461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.197237 6442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:48.197352 6442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "37fadca0b1a3a57cf5450385a556b945"
server_key_iv: "d0ea5931a5b425bf189d3ba94a673406"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.197429 6442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:48.208546 6442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:48.210950 6442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:48.211143 6442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:48.211681 6442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:48.211743 6442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.211787 6442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:48.211844 6442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.219332 6442 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:37647
I20260811 11:10:48.219723 6442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:48.220474 6590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:37647 every 8 connection(s)
I20260811 11:10:48.223536 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.212420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:50947 (local address 127.29.145.62:42489)
0811 11:10:48.212488 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.212490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.212799 (+ 309us) server_negotiation.cc:407] Connection header received
0811 11:10:48.213661 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.213663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.213678 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.213760 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.214704 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.215533 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.217162 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.217505 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.217620 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.220455 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.220456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.220457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.220465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.222164 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.222597 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.222599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.222599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.222681 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.222916 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.222917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.222917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.223034 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.223123 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.223393 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.223470 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:10:48.223695 6471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.212301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.1:50947)
0811 11:10:48.212657 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.212707 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.213546 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.213932 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.213937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.214299 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.214580 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.214587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.215662 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.215663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.217066 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.217069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.217638 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.217639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.217881 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.218542 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.218553 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.220327 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.222256 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.222263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.222265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.222502 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.222753 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.222754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.222754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.222855 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.223090 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.223095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.223281 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.223369 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.223500 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":93,"threads_started":1}
I20260811 11:10:48.224299 6591 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42489
I20260811 11:10:48.224390 6591 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:48.224584 6591 heartbeater.cc:511] Master 127.29.145.62:42489 requested a full tablet report, sending...
I20260811 11:10:48.225224 6356 ts_manager.cc:194] Registered new tserver with Master: 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647)
I20260811 11:10:48.226120 6356 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:50947
I20260811 11:10:48.229771 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 6442
I20260811 11:10:48.229912 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:48.230093 30276 external_mini_cluster.cc:1428] Setting key 1dd0f68a9b898f56df6f29af8f7c936f
I20260811 11:10:48.231698 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.225536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:35007 (local address 127.29.145.62:42489)
0811 11:10:48.225577 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.225579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.225719 (+ 140us) server_negotiation.cc:407] Connection header received
0811 11:10:48.225763 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.225764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.225777 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.225846 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.226250 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.226818 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.227559 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.227769 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.227849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.228827 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.228829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.228829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.228836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.230557 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.230930 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.230931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.230931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.230996 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.231230 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.231231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.231231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.231332 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.231387 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.231575 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.231640 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:10:48.231830 6598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.225472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.1:35007)
0811 11:10:48.225636 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.225647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.225710 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.225903 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.225905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.226082 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.226148 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.226150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.226917 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.226918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.227463 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.227465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.227848 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.227849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.227922 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.228409 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.228413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.228715 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.230639 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.230639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.230640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.230845 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.231074 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.231075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.231076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.231147 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.231404 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.231406 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.231470 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.231560 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.231673 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for oryx/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
I20260811 11:10:48.261068 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42489
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:48.350668 6601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:48.350884 6601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:48.350910 6601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:48.352069 6601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:48.352104 6601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:48.352169 6601 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:48.352186 6601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:48.353626 6601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:48.353869 6601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:48.354089 6601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:48.359190 6606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:48.359265 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:48.359270 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:48.359659 6601 server_base.cc:1034] running on GCE node
I20260811 11:10:48.359938 6601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:48.360221 6601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:48.361430 6601 hybrid_clock.cc:648] HybridClock initialized: now 1786446648361407 us; error 35 us; skew 500 ppm
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:48.364393 6601 init.cc:377] Logged in from keytab as oryx/127.29.145.2@KRBTEST.COM (short username oryx)
I20260811 11:10:48.364861 6601 webserver.cc:460] Webserver started at http://127.29.145.2:38025/ using document root <none> and password file <none>
I20260811 11:10:48.365126 6601 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:48.365199 6601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:48.365352 6601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:48.366508 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "9f1a5e152e9640b18678e22286e06a7c"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "0f2c6e7475f4dcadf52135d5aa608530"
server_key_iv: "8b08697818f4b84441083f29526e25bc"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.366889 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "9f1a5e152e9640b18678e22286e06a7c"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "0f2c6e7475f4dcadf52135d5aa608530"
server_key_iv: "8b08697818f4b84441083f29526e25bc"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.368323 6601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:10:48.369148 6617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.369344 6601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:48.369464 6601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "9f1a5e152e9640b18678e22286e06a7c"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "0f2c6e7475f4dcadf52135d5aa608530"
server_key_iv: "8b08697818f4b84441083f29526e25bc"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.369544 6601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:48.386852 6601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:48.389392 6601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:48.389575 6601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:48.390221 6601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:48.390285 6601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.390318 6601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:48.390349 6601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.397671 6601 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:45347
I20260811 11:10:48.397809 6745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:45347 every 8 connection(s)
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:48.398208 6601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:10:48.401547 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.390670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:33949 (local address 127.29.145.62:42489)
0811 11:10:48.390748 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.390751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.390965 (+ 214us) server_negotiation.cc:407] Connection header received
0811 11:10:48.391502 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.391504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.391516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.391593 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.392608 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.393423 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.395168 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.395510 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.395586 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.398714 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.398715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.398715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.398722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.399985 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.400519 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.400520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.400520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.400592 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.400835 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.400836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.400836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.400951 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.401006 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.401382 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.401483 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:10:48.401684 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.390651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.2:33949)
0811 11:10:48.390835 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.390875 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.391395 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.391724 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.391729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.392164 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.392477 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.392485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.393535 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.393537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.395081 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.395086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.395611 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.395612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.395798 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.396507 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.396521 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.398627 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.400041 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.400048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.400050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.400406 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.400668 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.400669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.400669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.400771 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.401012 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.401034 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.401166 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.401256 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.401470 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260811 11:10:48.402285 6746 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42489
I20260811 11:10:48.402372 6746 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:48.402575 6746 heartbeater.cc:511] Master 127.29.145.62:42489 requested a full tablet report, sending...
I20260811 11:10:48.403029 6356 ts_manager.cc:194] Registered new tserver with Master: 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347)
I20260811 11:10:48.403612 6356 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:33949
I20260811 11:10:48.407517 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 6601
I20260811 11:10:48.407657 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:48.407887 30276 external_mini_cluster.cc:1428] Setting key 2506445e5fdef687df0b1fff804aaf1a
I20260811 11:10:48.409555 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.403437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:56131 (local address 127.29.145.62:42489)
0811 11:10:48.403772 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.403774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.403782 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:48.403802 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.403802 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.403813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.403862 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.404286 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.404768 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.405630 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.405873 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.405949 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.406861 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.406863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.406863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.406869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.408374 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.408809 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.408810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.408811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.408863 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.409072 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.409072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.409072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.409208 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.409261 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.409430 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.409494 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315}
I20260811 11:10:48.409848 6754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.403346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.2:56131)
0811 11:10:48.403486 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.403498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.403552 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.403869 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.403871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.404083 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.404155 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.404157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.404879 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.404880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.405415 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.405416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.406000 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.406001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.406097 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.406487 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.406491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.406779 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.408469 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.408470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.408470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.408728 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.408932 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.408933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.408934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.409008 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.409274 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.409275 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.409324 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.409401 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.409524 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
I20260811 11:10:48.438637 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42489
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:48.522909 6757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:48.523111 6757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:48.523144 6757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:48.524415 6757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:48.524461 6757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:48.524482 6757 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:10:48.524498 6757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:48.526067 6757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.29.145.62:42489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:48.526306 6757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:48.526546 6757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:48.532105 6766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:48.532111 6763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:48.532111 6764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:48.532469 6757 server_base.cc:1034] running on GCE node
I20260811 11:10:48.532717 6757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:48.532953 6757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:48.534106 6757 hybrid_clock.cc:648] HybridClock initialized: now 1786446648534068 us; error 52 us; skew 500 ppm
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:48.536742 6757 init.cc:377] Logged in from keytab as oryx/127.29.145.3@KRBTEST.COM (short username oryx)
I20260811 11:10:48.537273 6757 webserver.cc:460] Webserver started at http://127.29.145.3:32843/ using document root <none> and password file <none>
I20260811 11:10:48.537509 6757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:48.537580 6757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:48.537741 6757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:48.538846 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "3f655d72d31b898ffb208c756ac7c3bb"
server_key_iv: "a272b4d7e84204d2280f1b152f861a78"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.539228 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "3f655d72d31b898ffb208c756ac7c3bb"
server_key_iv: "a272b4d7e84204d2280f1b152f861a78"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.540602 6757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:48.541416 6773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.541602 6757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:48.541694 6757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
format_stamp: "Formatted at 2026-08-11 11:10:48 on dist-test-slave-69rg"
server_key: "3f655d72d31b898ffb208c756ac7c3bb"
server_key_iv: "a272b4d7e84204d2280f1b152f861a78"
server_key_version: "encryptionkey@0"
I20260811 11:10:48.541791 6757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:48.548854 6757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:48.550864 6757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:48.551023 6757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:48.551599 6757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:48.551640 6757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:48.551662 6757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:48.551676 6757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:48.559263 6757 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:45489
I20260811 11:10:48.559340 6903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:45489 every 8 connection(s)
I20260811 11:10:48.559664 6757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:48.561983 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.552027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:41903 (local address 127.29.145.62:42489)
0811 11:10:48.552103 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.552107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.552307 (+ 200us) server_negotiation.cc:407] Connection header received
0811 11:10:48.552859 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.552860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.552874 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.552933 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.553534 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.554100 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.555155 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.555516 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.555605 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.558973 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.558975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.558975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.558984 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.560532 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.560976 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.560977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.560977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.561043 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.561312 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.561313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.561313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.561437 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.561509 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.561793 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.561894 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:48.562173 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.551979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.3:41903)
0811 11:10:48.552187 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.552227 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.552739 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.553005 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.553008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.553242 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.553432 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.553436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.554207 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.554208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.555056 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.555059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.555627 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.555628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.555793 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.556314 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.556322 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.558869 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.560640 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.560647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.560649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.560894 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.561123 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.561124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.561124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.561233 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.561513 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.561519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.561654 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.561755 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.561935 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":137,"threads_started":1}
I20260811 11:10:48.562683 6904 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42489
I20260811 11:10:48.562770 6904 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:48.562948 6904 heartbeater.cc:511] Master 127.29.145.62:42489 requested a full tablet report, sending...
I20260811 11:10:48.563402 6357 ts_manager.cc:194] Registered new tserver with Master: 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:48.563877 6357 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:41903
I20260811 11:10:48.564627 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 6757
I20260811 11:10:48.564742 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:48.564932 30276 external_mini_cluster.cc:1428] Setting key 154f7758f931a3a5d10aa65f40ede991
I20260811 11:10:48.565842 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:10:48.570035 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.563795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:60895 (local address 127.29.145.62:42489)
0811 11:10:48.563833 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.563835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.563910 (+ 75us) server_negotiation.cc:407] Connection header received
0811 11:10:48.563985 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.563986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.563998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.564040 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.564474 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.565174 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.566194 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.566397 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.566450 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.567481 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.567482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.567482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.567488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.568874 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.569235 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.569236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.569237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.569293 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.569503 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.569504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.569504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.569612 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.569685 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.569868 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.569943 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:48.570051 6911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.563728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.29.145.3:60895)
0811 11:10:48.563850 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.563860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.563918 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.564070 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.564071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.564301 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:48.564366 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.564368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.565277 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.565278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.566110 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.566113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.566468 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.566469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.566545 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.566975 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:48.566980 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:48.567387 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:48.568981 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.568982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.568983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.569154 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.569364 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:48.569365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:48.569366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:48.569431 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:48.569689 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:48.569690 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:48.569744 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.569840 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.569945 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:10:48.589414 6420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:48.583824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.29.145.62:42489)
0811 11:10:48.583885 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.583886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.583899 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:10:48.584018 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.584019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.584031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.584100 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.584644 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.585129 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.585935 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.586116 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.586171 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.589310 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58374: BlockingRecv error: recv got EOF from 127.0.0.1:58374 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 11:10:48 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446648, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:48.597065 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.590276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.29.145.62:42489)
0811 11:10:48.590335 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.590337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.590435 (+ 98us) server_negotiation.cc:407] Connection header received
0811 11:10:48.590509 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.590510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.590520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.590582 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:48.591070 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.591526 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.592209 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.592398 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.592475 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.594672 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:48.594674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:48.594675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:48.594683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:48.595919 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.596270 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.596271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.596272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.596342 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:48.596546 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:48.596547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:48.596547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:48.596662 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:48.596776 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.596879 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.596960 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 11:10:48.598902 6357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:48.600584 6357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:48.607383 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37647 (local address 127.0.0.1:44720)
0811 11:10:48.603564 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.603624 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.603694 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.604241 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.604246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.604249 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.604325 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.604327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605332 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.605333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.606360 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.606362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.607030 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.607031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.607131 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.607145 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.607238 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.607294 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260811 11:10:48.607664 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.29.145.1:37647)
0811 11:10:48.604046 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.604051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.604061 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:48.604099 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.604101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.604116 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.604232 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.604404 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605237 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.606453 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.606940 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.607028 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.607108 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.607514 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.607563 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":93,"threads_started":1}
I20260811 11:10:48.608474 6932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.0.0.1:46346)
0811 11:10:48.603857 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.603869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.603923 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.604306 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.604307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.604309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.604388 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.604391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605802 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.605803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.607147 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.607149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.608234 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.608235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.608299 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.608304 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.608360 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.608400 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260811 11:10:48.608399 6515 tablet_service.cc:1467] Processing CreateTablet for tablet 95f905a271d04d3282276842ace1195d (DEFAULT_TABLE table=test-table [id=33c9587937484fc5af928e1560f8a2ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:48.608717 6931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45489 (local address 127.0.0.1:36018)
0811 11:10:48.603564 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.603613 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.603694 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.604618 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.604618 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.604621 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.604655 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.604657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605955 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.605956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.607140 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.607143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.608493 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.608494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.608557 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.608561 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.608618 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.608658 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":4,"thread_start_us":83,"threads_started":1}
I20260811 11:10:48.608783 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.29.145.2:45347)
0811 11:10:48.604142 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.604148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.604159 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:48.604201 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.604203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.604221 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.604310 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.604472 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605688 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.607256 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.608075 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.608173 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.608252 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.608633 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.608675 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260811 11:10:48.608824 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.603920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.29.145.3:45489)
0811 11:10:48.604440 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.604445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.604457 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:10:48.604495 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.604498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.604516 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.604614 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.604719 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.605864 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.607242 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.608194 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.608382 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.608459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.608658 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.608703 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":91,"threads_started":1}
I20260811 11:10:48.608878 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f905a271d04d3282276842ace1195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:48.609391 6832 tablet_service.cc:1467] Processing CreateTablet for tablet 95f905a271d04d3282276842ace1195d (DEFAULT_TABLE table=test-table [id=33c9587937484fc5af928e1560f8a2ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:48.609643 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f905a271d04d3282276842ace1195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:48.612637 6936 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Bootstrap starting.
I20260811 11:10:48.612653 6937 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Bootstrap starting.
I20260811 11:10:48.613306 6937 tablet_bootstrap.cc:654] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:48.613396 6936 tablet_bootstrap.cc:654] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:48.613345 6673 tablet_service.cc:1467] Processing CreateTablet for tablet 95f905a271d04d3282276842ace1195d (DEFAULT_TABLE table=test-table [id=33c9587937484fc5af928e1560f8a2ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:48.613652 6937 log.cc:824] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:48.613721 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f905a271d04d3282276842ace1195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:48.613766 6936 log.cc:824] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:48.614562 6937 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: No bootstrap required, opened a new log
I20260811 11:10:48.614571 6936 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: No bootstrap required, opened a new log
I20260811 11:10:48.614650 6936 ts_tablet_manager.cc:1397] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:48.614667 6937 ts_tablet_manager.cc:1397] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:10:48.616793 6940 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Bootstrap starting.
I20260811 11:10:48.616837 6937 raft_consensus.cc:348] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.616837 6936 raft_consensus.cc:348] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.617027 6937 raft_consensus.cc:374] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:48.617027 6936 raft_consensus.cc:374] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:48.617079 6936 raft_consensus.cc:729] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Initialized, Role: FOLLOWER
I20260811 11:10:48.617079 6937 raft_consensus.cc:729] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Initialized, Role: FOLLOWER
I20260811 11:10:48.617203 6937 consensus_queue.cc:260] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.617225 6936 consensus_queue.cc:260] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.617506 6940 tablet_bootstrap.cc:654] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:48.617515 6937 ts_tablet_manager.cc:1428] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:10:48.617540 6591 heartbeater.cc:503] Master 127.29.145.62:42489 was elected leader, sending a full tablet report...
I20260811 11:10:48.617707 6904 heartbeater.cc:503] Master 127.29.145.62:42489 was elected leader, sending a full tablet report...
I20260811 11:10:48.617888 6940 log.cc:824] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:48.618309 6936 ts_tablet_manager.cc:1428] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 11:10:48.619118 6940 tablet_bootstrap.cc:492] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: No bootstrap required, opened a new log
I20260811 11:10:48.619174 6940 ts_tablet_manager.cc:1397] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:48.620553 6940 raft_consensus.cc:348] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.620677 6940 raft_consensus.cc:374] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:48.620699 6940 raft_consensus.cc:729] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Initialized, Role: FOLLOWER
I20260811 11:10:48.620823 6940 consensus_queue.cc:260] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.621086 6940 ts_tablet_manager.cc:1428] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 11:10:48.621172 6746 heartbeater.cc:503] Master 127.29.145.62:42489 was elected leader, sending a full tablet report...
W20260811 11:10:48.648823 6747 tablet.cc:2369] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:48.720521 6592 tablet.cc:2369] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:10:48.810397 6905 tablet.cc:2369] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:10:48.987432 6941 raft_consensus.cc:482] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:48.987600 6941 raft_consensus.cc:504] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.988065 6941 leader_election.cc:288] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347)
I20260811 11:10:48.991000 6944 raft_consensus.cc:482] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:48.991117 6944 raft_consensus.cc:504] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.991725 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.988275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37647 (local address 127.29.145.3:54923)
0811 11:10:48.988338 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.988354 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.988408 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.988570 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.988572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.988575 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.988663 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.988665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.989396 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.989397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.990675 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.990678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.991368 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.991370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.991486 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.991492 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.991549 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.991601 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":7}
I20260811 11:10:48.991755 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.988514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:45505 (local address 127.29.145.2:45347)
0811 11:10:48.988575 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.988577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.988586 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:48.988614 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.988615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.988625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.988694 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.988904 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.989442 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.990803 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.991327 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.991398 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.991407 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.991633 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.991681 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:10:48.991787 6944 leader_election.cc:288] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:48.992000 6945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.988292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.29.145.3:45505)
0811 11:10:48.988511 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.988520 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.988567 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.988723 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.988724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.988726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.988830 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.988833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.989514 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.989515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.990675 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.990678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.991445 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.991446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.991527 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.991532 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.991834 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.991882 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":14,"thread_start_us":128,"threads_started":1}
I20260811 11:10:48.992209 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.988378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:54923 (local address 127.29.145.1:37647)
0811 11:10:48.988429 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.988431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.988441 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:48.988472 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.988473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.988484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.988546 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.988746 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.989312 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.990772 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.991286 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.991372 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.991383 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.992069 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.992139 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:10:48.992295 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c" is_pre_election: true
I20260811 11:10:48.992408 6693 raft_consensus.cc:2454] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 0.
I20260811 11:10:48.992727 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" is_pre_election: true
I20260811 11:10:48.992868 6539 raft_consensus.cc:2454] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 0.
I20260811 11:10:48.993045 6775 leader_election.cc:302] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:48.993283 6941 raft_consensus.cc:2781] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:48.993357 6941 raft_consensus.cc:482] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:48.993389 6941 raft_consensus.cc:3037] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:48.994339 6941 raft_consensus.cc:504] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.994539 6941 leader_election.cc:288] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 election: Requested vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347)
I20260811 11:10:48.994621 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c"
I20260811 11:10:48.994697 6693 raft_consensus.cc:3037] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:48.994681 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
I20260811 11:10:48.994755 6539 raft_consensus.cc:3037] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:48.994884 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.991734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:58717 (local address 127.29.145.3:45489)
0811 11:10:48.991771 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.991773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.991945 (+ 172us) server_negotiation.cc:407] Connection header received
0811 11:10:48.992047 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.992048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.992055 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.992084 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.992324 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.992958 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.994088 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.994456 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.994527 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.994535 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.994794 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.994831 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 11:10:48.995056 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.991586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45489 (local address 127.29.145.2:58717)
0811 11:10:48.991879 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.991891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.991975 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.992135 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.992136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.992137 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.992230 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.992232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.993052 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.993052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.994004 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.994007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.994640 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.994641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.994708 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.994712 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.994943 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.994983 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":64,"threads_started":1}
I20260811 11:10:48.995384 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" is_pre_election: true
I20260811 11:10:48.995440 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.991588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37647 (local address 127.29.145.2:52123)
0811 11:10:48.991914 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:48.991925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:48.991976 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:48.992412 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:48.992413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:48.992416 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:48.992469 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.992471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.993328 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.993329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.994449 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:48.994453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.995189 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.995190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:48.995267 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.995271 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:10:48.995321 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 11:10:48.995362 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":79,"threads_started":1}
I20260811 11:10:48.995493 6853 raft_consensus.cc:2379] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1a5e152e9640b18678e22286e06a7c in current term 1: Already voted for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in this term.
I20260811 11:10:48.995563 6693 raft_consensus.cc:2454] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 1.
I20260811 11:10:48.995563 6539 raft_consensus.cc:2454] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 1.
I20260811 11:10:48.995733 6775 leader_election.cc:302] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:48.995754 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:48.991765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:52123 (local address 127.29.145.1:37647)
0811 11:10:48.992280 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:48.992282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:48.992290 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:48.992311 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:48.992312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:48.992321 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:48.992373 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:48.992553 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.993237 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.994552 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:48.995069 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:48.995153 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:48.995161 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:48.995520 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 11:10:48.995556 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":62,"threads_started":1}
I20260811 11:10:48.995807 6941 raft_consensus.cc:2781] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:48.995954 6941 raft_consensus.cc:686] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Running, Role: LEADER
I20260811 11:10:48.995924 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f905a271d04d3282276842ace1195d" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" is_pre_election: true
I20260811 11:10:48.996012 6539 raft_consensus.cc:2379] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1a5e152e9640b18678e22286e06a7c in current term 1: Already voted for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in this term.
I20260811 11:10:48.996050 6941 consensus_queue.cc:237] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } }
I20260811 11:10:48.996263 6621 leader_election.cc:302] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1a5e152e9640b18678e22286e06a7c; no voters: 08c1f2e18ce74952ac94e4cb32576fa8, 72dcf3dd0eb2471dbb15cdcf6d6f3561
I20260811 11:10:48.996405 6944 raft_consensus.cc:2726] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 11:10:48.996941 6357 catalog_manager.cc:5495] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:49.051770 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.049072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.29.145.3:45489)
0811 11:10:49.049155 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.049157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.049168 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:49.049215 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.049216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.049234 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.049302 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.049536 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.050099 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.050916 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.051101 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.051161 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.051359 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.051482 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.051546 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.051640 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.051659 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:10:49.053718 6539 raft_consensus.cc:1261] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Refusing update from remote peer 08c1f2e18ce74952ac94e4cb32576fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.053748 6693 raft_consensus.cc:1261] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Refusing update from remote peer 08c1f2e18ce74952ac94e4cb32576fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.054108 6949 consensus_queue.cc:1035] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.054287 6941 consensus_queue.cc:1035] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.057468 6953 mvcc.cc:205] Tried to move back new op lower bound from 7317285474520809472 to 7317285474288132096. Current Snapshot: MvccSnapshot[applied={T|T < 7317285474520809472}]
I20260811 11:10:49.059717 6356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.059819 6356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.060747 6356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Sending DeleteTablet for 3 replicas of tablet 95f905a271d04d3282276842ace1195d
I20260811 11:10:49.061107 6515 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f905a271d04d3282276842ace1195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:44720
I20260811 11:10:49.061213 6673 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f905a271d04d3282276842ace1195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260811 11:10:49.061357 6832 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f905a271d04d3282276842ace1195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:36018
I20260811 11:10:49.061381 6965 tablet_replica.cc:330] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: stopping tablet replica
I20260811 11:10:49.061491 6965 raft_consensus.cc:2229] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.061600 6965 raft_consensus.cc:2258] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.061547 6356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:49.061683 6356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:49.062024 6967 tablet_replica.cc:330] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: stopping tablet replica
I20260811 11:10:49.062121 6967 raft_consensus.cc:2229] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:49.062219 6967 raft_consensus.cc:2258] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.062342 6965 ts_tablet_manager.cc:1905] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.063048 6967 ts_tablet_manager.cc:1905] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.063627 6515 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4c3916ed1242839387a89febffb8bd (DEFAULT_TABLE table=test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.063843 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4c3916ed1242839387a89febffb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.064270 6965 ts_tablet_manager.cc:1918] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.064344 6965 log.cc:1196] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/95f905a271d04d3282276842ace1195d
I20260811 11:10:49.064613 6965 ts_tablet_manager.cc:1939] T 95f905a271d04d3282276842ace1195d P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting consensus metadata
I20260811 11:10:49.065217 6967 ts_tablet_manager.cc:1918] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.065289 6967 log.cc:1196] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/95f905a271d04d3282276842ace1195d
I20260811 11:10:49.065383 6832 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4c3916ed1242839387a89febffb8bd (DEFAULT_TABLE table=test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.065493 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4c3916ed1242839387a89febffb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.065567 6967 ts_tablet_manager.cc:1939] T 95f905a271d04d3282276842ace1195d P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting consensus metadata
I20260811 11:10:49.065930 6937 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Bootstrap starting.
I20260811 11:10:49.066340 6937 tablet_bootstrap.cc:654] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.066547 6936 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Bootstrap starting.
I20260811 11:10:49.066720 6338 catalog_manager.cc:4841] TS 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489): tablet 95f905a271d04d3282276842ace1195d (table test-table [id=33c9587937484fc5af928e1560f8a2ff]) successfully deleted
I20260811 11:10:49.066900 6342 catalog_manager.cc:4841] TS 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647): tablet 95f905a271d04d3282276842ace1195d (table test-table [id=33c9587937484fc5af928e1560f8a2ff]) successfully deleted
I20260811 11:10:49.066987 6936 tablet_bootstrap.cc:654] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.061903 6966 tablet_replica.cc:330] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: stopping tablet replica
I20260811 11:10:49.067144 6966 raft_consensus.cc:2229] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.065313 6673 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4c3916ed1242839387a89febffb8bd (DEFAULT_TABLE table=test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.067297 6966 raft_consensus.cc:2258] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.067481 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4c3916ed1242839387a89febffb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.067710 6936 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: No bootstrap required, opened a new log
I20260811 11:10:49.067760 6936 ts_tablet_manager.cc:1397] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.067919 6936 raft_consensus.cc:348] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.067993 6936 raft_consensus.cc:374] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.068022 6936 raft_consensus.cc:729] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.068063 6936 consensus_queue.cc:260] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.068162 6936 ts_tablet_manager.cc:1428] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:49.068306 6966 ts_tablet_manager.cc:1905] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.069011 6940 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Bootstrap starting.
I20260811 11:10:49.069473 6940 tablet_bootstrap.cc:654] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.069759 6937 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: No bootstrap required, opened a new log
I20260811 11:10:49.069839 6937 ts_tablet_manager.cc:1397] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 11:10:49.069999 6937 raft_consensus.cc:348] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.070060 6937 raft_consensus.cc:374] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.070073 6937 raft_consensus.cc:729] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.070148 6937 consensus_queue.cc:260] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.070279 6937 ts_tablet_manager.cc:1428] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:49.070679 6966 ts_tablet_manager.cc:1918] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.070782 6966 log.cc:1196] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/95f905a271d04d3282276842ace1195d
I20260811 11:10:49.070840 6940 tablet_bootstrap.cc:492] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: No bootstrap required, opened a new log
I20260811 11:10:49.070894 6940 ts_tablet_manager.cc:1397] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:49.071079 6940 raft_consensus.cc:348] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.071151 6966 ts_tablet_manager.cc:1939] T 95f905a271d04d3282276842ace1195d P 9f1a5e152e9640b18678e22286e06a7c: Deleting consensus metadata
I20260811 11:10:49.071175 6940 raft_consensus.cc:374] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.071341 6940 raft_consensus.cc:729] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.071414 6940 consensus_queue.cc:260] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.071552 6940 ts_tablet_manager.cc:1428] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:49.071743 6338 catalog_manager.cc:4841] TS 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347): tablet 95f905a271d04d3282276842ace1195d (table test-table [id=33c9587937484fc5af928e1560f8a2ff]) successfully deleted
I20260811 11:10:49.453660 6942 raft_consensus.cc:482] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:49.453786 6942 raft_consensus.cc:504] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.454237 6942 leader_election.cc:288] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.457356 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.454674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:53295 (local address 127.29.145.2:45347)
0811 11:10:49.454758 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.454760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.454794 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:10:49.454871 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.454872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.454883 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.454968 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:49.455126 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.455651 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.456643 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.457002 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.457094 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.457104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.457248 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.457284 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:10:49.457376 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.454558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.29.145.1:53295)
0811 11:10:49.454719 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.454733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.454794 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.454976 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.454978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.454981 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:49.455048 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.455050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.455753 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.455754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.456543 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.456545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.457093 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.457094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.457170 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.457175 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.457242 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.457284 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260811 11:10:49.457468 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.454788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45489 (local address 127.29.145.1:46909)
0811 11:10:49.454943 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.454954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.455007 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.455165 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.455166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.455168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:10:49.455242 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.455244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.455907 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.455908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.456635 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.456637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.457191 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.457192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.457272 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.457275 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.457342 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.457382 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260811 11:10:49.457453 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.454887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:46909 (local address 127.29.145.3:45489)
0811 11:10:49.454950 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.454954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.455001 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:10:49.455066 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.455067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.455080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.455160 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:10:49.455325 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.455795 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.456737 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.457086 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.457180 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.457189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.457350 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.457387 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:10:49.457659 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4c3916ed1242839387a89febffb8bd" candidate_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" is_pre_election: true
I20260811 11:10:49.457778 6853 raft_consensus.cc:2454] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72dcf3dd0eb2471dbb15cdcf6d6f3561 in term 0.
I20260811 11:10:49.457798 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4c3916ed1242839387a89febffb8bd" candidate_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c" is_pre_election: true
I20260811 11:10:49.457942 6693 raft_consensus.cc:2454] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72dcf3dd0eb2471dbb15cdcf6d6f3561 in term 0.
I20260811 11:10:49.458022 6462 leader_election.cc:302] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 72dcf3dd0eb2471dbb15cdcf6d6f3561; no voters:
I20260811 11:10:49.458166 6942 raft_consensus.cc:2781] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:49.458233 6942 raft_consensus.cc:482] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:49.458267 6942 raft_consensus.cc:3037] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.459004 6942 raft_consensus.cc:504] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.459158 6942 leader_election.cc:288] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.459465 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4c3916ed1242839387a89febffb8bd" candidate_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
I20260811 11:10:49.459462 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4c3916ed1242839387a89febffb8bd" candidate_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c"
I20260811 11:10:49.459547 6693 raft_consensus.cc:3037] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.459548 6853 raft_consensus.cc:3037] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.460306 6693 raft_consensus.cc:2454] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72dcf3dd0eb2471dbb15cdcf6d6f3561 in term 1.
I20260811 11:10:49.460474 6853 raft_consensus.cc:2454] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72dcf3dd0eb2471dbb15cdcf6d6f3561 in term 1.
I20260811 11:10:49.460508 6462 leader_election.cc:302] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72dcf3dd0eb2471dbb15cdcf6d6f3561, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:49.460600 6942 raft_consensus.cc:2781] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:49.460772 6942 raft_consensus.cc:686] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 LEADER]: Becoming Leader. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Running, Role: LEADER
I20260811 11:10:49.460880 6942 consensus_queue.cc:237] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.461674 6356 catalog_manager.cc:5495] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1). New cstate: current_term: 1 leader_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446647, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.62@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:49.546311 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.538797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.29.145.62:42489)
0811 11:10:49.539061 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.539063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.539074 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:49.539226 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.539226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.539237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.539309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:49.539807 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.540320 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.541040 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.541235 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.541324 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.543607 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:49.543608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:49.543609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:49.543616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:49.544959 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:49.545397 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:49.545398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:49.545398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:49.545460 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:49.545687 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:49.545688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:49.545689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:49.545807 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:49.545917 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.546077 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.546191 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":74,"threads_started":1}
I20260811 11:10:49.546372 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.538697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42489 (local address 127.0.0.1:36418)
0811 11:10:49.538959 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.538979 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.539129 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.539325 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.539327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.539656 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:49.539704 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.539706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.540407 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.540408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.540949 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.540950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.541326 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.541326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.541380 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.542014 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:49.542020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:49.543509 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:49.545063 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:49.545071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:49.545073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:49.545306 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:49.545554 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:49.545555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:49.545555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:49.545618 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:49.545905 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:49.545913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:49.545962 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.546051 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.546250 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":89,"thread_start_us":154,"threads_started":1}
I20260811 11:10:49.548616 6356 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:36418:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:10:49.548875 6356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:49.550531 6515 tablet_service.cc:1467] Processing CreateTablet for tablet 36baff7787ec4bdb9f4975b18f0dd880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca8f12e9c42648c78134acace3e987d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:49.550549 6673 tablet_service.cc:1467] Processing CreateTablet for tablet 36baff7787ec4bdb9f4975b18f0dd880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca8f12e9c42648c78134acace3e987d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:49.550716 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36baff7787ec4bdb9f4975b18f0dd880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.550750 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36baff7787ec4bdb9f4975b18f0dd880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.552002 6940 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: Bootstrap starting.
I20260811 11:10:49.552002 6937 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Bootstrap starting.
I20260811 11:10:49.552282 6832 tablet_service.cc:1467] Processing CreateTablet for tablet 36baff7787ec4bdb9f4975b18f0dd880 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ca8f12e9c42648c78134acace3e987d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:10:49.552444 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36baff7787ec4bdb9f4975b18f0dd880. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.552465 6940 tablet_bootstrap.cc:654] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.552465 6937 tablet_bootstrap.cc:654] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.553130 6937 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: No bootstrap required, opened a new log
I20260811 11:10:49.553203 6937 ts_tablet_manager.cc:1397] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.553409 6940 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: No bootstrap required, opened a new log
I20260811 11:10:49.553373 6937 raft_consensus.cc:348] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.553452 6937 raft_consensus.cc:374] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.553462 6940 ts_tablet_manager.cc:1397] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:49.553478 6937 raft_consensus.cc:729] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.553539 6937 consensus_queue.cc:260] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.553602 6940 raft_consensus.cc:348] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.553647 6937 ts_tablet_manager.cc:1428] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:49.553678 6940 raft_consensus.cc:374] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.553701 6940 raft_consensus.cc:729] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.553745 6940 consensus_queue.cc:260] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.553853 6940 ts_tablet_manager.cc:1428] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:49.553782 6942 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.553903 6942 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.553999 6936 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: Bootstrap starting.
I20260811 11:10:49.554078 6944 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.554167 6944 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.554368 6936 tablet_bootstrap.cc:654] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.554955 6936 tablet_bootstrap.cc:492] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: No bootstrap required, opened a new log
I20260811 11:10:49.555022 6936 ts_tablet_manager.cc:1397] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.555142 6936 raft_consensus.cc:348] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.555215 6936 raft_consensus.cc:374] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.555238 6936 raft_consensus.cc:729] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.555289 6936 consensus_queue.cc:260] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.555383 6936 ts_tablet_manager.cc:1428] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:49.555433 6949 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.555560 6949 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.573320 6944 raft_consensus.cc:482] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:49.573439 6944 raft_consensus.cc:504] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.573645 6944 leader_election.cc:288] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.573909 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36baff7787ec4bdb9f4975b18f0dd880" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" is_pre_election: true
I20260811 11:10:49.573990 6853 raft_consensus.cc:2454] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 0.
I20260811 11:10:49.573926 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36baff7787ec4bdb9f4975b18f0dd880" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" is_pre_election: true
I20260811 11:10:49.574060 6539 raft_consensus.cc:2454] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 0.
I20260811 11:10:49.574152 6618 leader_election.cc:302] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:49.574260 6944 raft_consensus.cc:2781] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:49.574296 6944 raft_consensus.cc:482] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:49.574330 6944 raft_consensus.cc:3037] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.575042 6944 raft_consensus.cc:504] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.575204 6944 leader_election.cc:288] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 election: Requested vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.575429 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36baff7787ec4bdb9f4975b18f0dd880" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
I20260811 11:10:49.575428 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36baff7787ec4bdb9f4975b18f0dd880" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
I20260811 11:10:49.575515 6539 raft_consensus.cc:3037] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.575515 6853 raft_consensus.cc:3037] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.576299 6539 raft_consensus.cc:2454] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 1.
I20260811 11:10:49.576309 6853 raft_consensus.cc:2454] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 1.
I20260811 11:10:49.576522 6621 leader_election.cc:302] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72dcf3dd0eb2471dbb15cdcf6d6f3561, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:49.576678 6944 raft_consensus.cc:2781] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:49.576817 6944 raft_consensus.cc:686] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [term 1 LEADER]: Becoming Leader. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Running, Role: LEADER
I20260811 11:10:49.576893 6944 consensus_queue.cc:237] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.577301 6986 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: TxnStatusTablet state changed. Reason: New leader 9f1a5e152e9640b18678e22286e06a7c. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.577397 6986 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:49.577538 6989 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:10:49.577675 6989 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:10:49.577759 6356 catalog_manager.cc:5495] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2). New cstate: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:49.585027 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.582096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.0.0.1:49824)
0811 11:10:49.582197 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.582210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.582268 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.582432 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.582433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.582434 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:49.582480 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.582482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.583407 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.583408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.584006 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.584008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.584491 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.584492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.584544 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.584561 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.584879 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.584888 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.584928 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.584972 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 11:10:49.585028 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.582183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.29.145.2:45347)
0811 11:10:49.582225 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.582228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.582255 (+ 27us) server_negotiation.cc:407] Connection header received
0811 11:10:49.582328 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.582329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.582349 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.582433 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.582536 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.583304 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.584101 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.584402 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.584476 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.584679 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.584803 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.584883 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.584933 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.584955 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:10:49.586562 6539 raft_consensus.cc:1261] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Refusing update from remote peer 9f1a5e152e9640b18678e22286e06a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.586623 6853 raft_consensus.cc:1261] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Refusing update from remote peer 9f1a5e152e9640b18678e22286e06a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.586843 6944 consensus_queue.cc:1035] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.586990 6986 consensus_queue.cc:1035] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.587585 6942 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: TxnStatusTablet state changed. Reason: New leader 9f1a5e152e9640b18678e22286e06a7c. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.587687 6942 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.588321 6949 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: TxnStatusTablet state changed. Reason: New leader 9f1a5e152e9640b18678e22286e06a7c. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.588434 6949 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.588732 6986 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.588822 6986 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:49.589206 6994 mvcc.cc:205] Tried to move back new op lower bound from 7317285476704399360 to 7317285476667789312. Current Snapshot: MvccSnapshot[applied={T|T < 7317285476704399360}]
I20260811 11:10:49.589308 6942 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.589398 6942 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.590394 6986 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.590489 6986 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 9f1a5e152e9640b18678e22286e06a7c: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:10:49.590739 6996 mvcc.cc:205] Tried to move back new op lower bound from 7317285476704399360 to 7317285476667789312. Current Snapshot: MvccSnapshot[applied={T|T < 7317285476704399360}]
I20260811 11:10:49.590835 6949 tablet_replica.cc:439] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } } }
I20260811 11:10:49.590907 6949 tablet_replica.cc:442] T 36baff7787ec4bdb9f4975b18f0dd880 P 08c1f2e18ce74952ac94e4cb32576fa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:10:49.597009 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.591146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.29.145.1:37647)
0811 11:10:49.593976 (+ 2830us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.593978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.593987 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:49.594029 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.594029 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.594051 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.594141 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.594315 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.595093 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.595904 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.596225 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.596299 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.596608 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.596751 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.596829 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.596874 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.596898 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2808,"thread_start_us":58,"threads_started":1}
I20260811 11:10:49.602829 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.599840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:35961 (local address 127.29.145.2:45347)
0811 11:10:49.599912 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.599914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.600071 (+ 157us) server_negotiation.cc:407] Connection header received
0811 11:10:49.600177 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.600178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.600189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.600256 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.600471 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.600997 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.602078 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.602292 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.602388 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.602586 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.602612 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.602653 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.602756 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.602774 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:10:49.602895 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.599741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.29.145.1:35961)
0811 11:10:49.599987 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.599998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.600099 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.600292 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.600293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.600295 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:49.600357 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.600358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.601102 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.601103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.601973 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.601976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.602403 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.602405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.602482 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.602499 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.602690 (+ 191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.602692 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.602753 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.602799 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":100,"threads_started":1}
I20260811 11:10:49.605965 6853 raft_consensus.cc:1261] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Refusing update from remote peer 72dcf3dd0eb2471dbb15cdcf6d6f3561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.605978 6693 raft_consensus.cc:1261] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Refusing update from remote peer 72dcf3dd0eb2471dbb15cdcf6d6f3561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.606236 6942 consensus_queue.cc:1035] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.606405 6974 consensus_queue.cc:1035] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.616750 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.613437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37647 (local address 127.29.145.2:53555)
0811 11:10:49.613644 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.613663 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.613721 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.613884 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.613885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.613886 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:49.613960 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.613961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.614854 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.614856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.615851 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.615855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.616325 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.616327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.616388 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.616402 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.616602 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.616605 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.616638 (+ 33us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.616677 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260811 11:10:49.616745 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.613466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:53555 (local address 127.29.145.1:37647)
0811 11:10:49.613512 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.613514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.613762 (+ 248us) server_negotiation.cc:407] Connection header received
0811 11:10:49.613790 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.613791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.613801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.613880 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.614024 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.614768 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.615928 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.616226 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.616299 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.616488 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.616522 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.616574 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.616649 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.616668 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:10:49.625458 6357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.625566 6357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.626369 6357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Sending DeleteTablet for 3 replicas of tablet 2f4c3916ed1242839387a89febffb8bd
I20260811 11:10:49.626808 6515 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4c3916ed1242839387a89febffb8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:44720
I20260811 11:10:49.630597 6832 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4c3916ed1242839387a89febffb8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:36018
I20260811 11:10:49.630872 7021 tablet_replica.cc:330] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: stopping tablet replica
I20260811 11:10:49.630874 6673 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4c3916ed1242839387a89febffb8bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260811 11:10:49.631013 7021 raft_consensus.cc:2229] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.631143 7021 raft_consensus.cc:2258] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.631451 7021 ts_tablet_manager.cc:1905] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.633424 7021 ts_tablet_manager.cc:1918] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.633567 7021 log.cc:1196] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/2f4c3916ed1242839387a89febffb8bd
I20260811 11:10:49.633945 7021 ts_tablet_manager.cc:1939] T 2f4c3916ed1242839387a89febffb8bd P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting consensus metadata
I20260811 11:10:49.634313 7022 tablet_replica.cc:330] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: stopping tablet replica
I20260811 11:10:49.634353 6338 catalog_manager.cc:4841] TS 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489): tablet 2f4c3916ed1242839387a89febffb8bd (table test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]) successfully deleted
I20260811 11:10:49.634379 7022 raft_consensus.cc:2229] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.634418 7022 raft_consensus.cc:2258] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.634666 7022 ts_tablet_manager.cc:1905] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.636291 7022 ts_tablet_manager.cc:1918] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.636355 7022 log.cc:1196] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/2f4c3916ed1242839387a89febffb8bd
I20260811 11:10:49.636598 7022 ts_tablet_manager.cc:1939] T 2f4c3916ed1242839387a89febffb8bd P 9f1a5e152e9640b18678e22286e06a7c: Deleting consensus metadata
I20260811 11:10:49.636819 7018 tablet_replica.cc:330] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: stopping tablet replica
I20260811 11:10:49.636885 7018 raft_consensus.cc:2229] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:49.636981 7018 raft_consensus.cc:2258] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.637238 7018 ts_tablet_manager.cc:1905] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.638314 6338 catalog_manager.cc:4841] TS 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347): tablet 2f4c3916ed1242839387a89febffb8bd (table test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]) successfully deleted
I20260811 11:10:49.639339 7018 ts_tablet_manager.cc:1918] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.639405 7018 log.cc:1196] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/2f4c3916ed1242839387a89febffb8bd
I20260811 11:10:49.639640 7018 ts_tablet_manager.cc:1939] T 2f4c3916ed1242839387a89febffb8bd P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting consensus metadata
I20260811 11:10:49.639956 6342 catalog_manager.cc:4841] TS 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647): tablet 2f4c3916ed1242839387a89febffb8bd (table test-table [id=7e49ea4d7bb541e4b8624e24c3badfa9]) successfully deleted
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[6300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:10:49.665705 6982 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:49.657979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.29.145.62:42489)
0811 11:10:49.658052 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.658055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.658200 (+ 145us) server_negotiation.cc:407] Connection header received
0811 11:10:49.658244 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.658246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.658262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.658336 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:49.659067 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.659982 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.661100 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.661462 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.661543 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.665618 (+ 4075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36430: BlockingRecv error: recv got EOF from 127.0.0.1:36430 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[6300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM
I20260811 11:10:49.679234 6982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.670139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.29.145.62:42489)
0811 11:10:49.670192 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.670194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.670275 (+ 81us) server_negotiation.cc:407] Connection header received
0811 11:10:49.670332 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.670333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.670344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.670415 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:49.671034 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.671830 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.672837 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.673150 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.673265 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.676065 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:49.676066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:49.676067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:49.676074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:49.677778 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:49.678229 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:49.678231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:49.678231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:49.678297 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:49.678529 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:49.678530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:49.678531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:49.678684 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:49.678795 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.678938 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.679024 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:10:49.681191 6357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:49.681371 6357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:49.682992 6673 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8d30044d4542f6846636cc41d8b231 (DEFAULT_TABLE table=test-table [id=c25777820c4244428087b87c7a4f8454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.683142 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8d30044d4542f6846636cc41d8b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.684708 6515 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8d30044d4542f6846636cc41d8b231 (DEFAULT_TABLE table=test-table [id=c25777820c4244428087b87c7a4f8454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.684839 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8d30044d4542f6846636cc41d8b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.685946 6940 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Bootstrap starting.
I20260811 11:10:49.686302 6937 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Bootstrap starting.
I20260811 11:10:49.686412 6940 tablet_bootstrap.cc:654] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.686749 6937 tablet_bootstrap.cc:654] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.686786 6832 tablet_service.cc:1467] Processing CreateTablet for tablet 9b8d30044d4542f6846636cc41d8b231 (DEFAULT_TABLE table=test-table [id=c25777820c4244428087b87c7a4f8454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.686918 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b8d30044d4542f6846636cc41d8b231. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.687554 6940 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: No bootstrap required, opened a new log
I20260811 11:10:49.687615 6940 ts_tablet_manager.cc:1397] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:10:49.687783 6940 raft_consensus.cc:348] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.687855 6940 raft_consensus.cc:374] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.687878 6940 raft_consensus.cc:729] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.687925 6940 consensus_queue.cc:260] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.688016 6940 ts_tablet_manager.cc:1428] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:49.688194 6936 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Bootstrap starting.
I20260811 11:10:49.688517 6936 tablet_bootstrap.cc:654] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.689324 6936 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: No bootstrap required, opened a new log
I20260811 11:10:49.689381 6936 ts_tablet_manager.cc:1397] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.689486 6936 raft_consensus.cc:348] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.689553 6936 raft_consensus.cc:374] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.689574 6936 raft_consensus.cc:729] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.689613 6936 consensus_queue.cc:260] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.689702 6936 ts_tablet_manager.cc:1428] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:49.690814 6937 tablet_bootstrap.cc:492] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: No bootstrap required, opened a new log
I20260811 11:10:49.690868 6937 ts_tablet_manager.cc:1397] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 11:10:49.691010 6937 raft_consensus.cc:348] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.691077 6937 raft_consensus.cc:374] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.691098 6937 raft_consensus.cc:729] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.691147 6937 consensus_queue.cc:260] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.691237 6937 ts_tablet_manager.cc:1428] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:49.701402 7002 raft_consensus.cc:482] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:49.701493 7002 raft_consensus.cc:504] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.701666 7002 leader_election.cc:288] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.701999 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8d30044d4542f6846636cc41d8b231" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" is_pre_election: true
I20260811 11:10:49.702019 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8d30044d4542f6846636cc41d8b231" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" is_pre_election: true
I20260811 11:10:49.702087 6539 raft_consensus.cc:2454] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 0.
I20260811 11:10:49.702107 6853 raft_consensus.cc:2454] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 0.
I20260811 11:10:49.702296 6621 leader_election.cc:302] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72dcf3dd0eb2471dbb15cdcf6d6f3561, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:49.702468 7002 raft_consensus.cc:2781] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:49.702514 7002 raft_consensus.cc:482] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:49.702543 7002 raft_consensus.cc:3037] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.703486 7002 raft_consensus.cc:504] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.703622 7002 leader_election.cc:288] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 election: Requested vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489)
I20260811 11:10:49.703840 6853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8d30044d4542f6846636cc41d8b231" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c1f2e18ce74952ac94e4cb32576fa8"
I20260811 11:10:49.703861 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b8d30044d4542f6846636cc41d8b231" candidate_uuid: "9f1a5e152e9640b18678e22286e06a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
I20260811 11:10:49.703918 6853 raft_consensus.cc:3037] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.703933 6539 raft_consensus.cc:3037] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.704726 6539 raft_consensus.cc:2454] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 1.
I20260811 11:10:49.704730 6853 raft_consensus.cc:2454] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1a5e152e9640b18678e22286e06a7c in term 1.
I20260811 11:10:49.704893 6621 leader_election.cc:302] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72dcf3dd0eb2471dbb15cdcf6d6f3561, 9f1a5e152e9640b18678e22286e06a7c; no voters:
I20260811 11:10:49.705098 7002 raft_consensus.cc:2781] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:49.705149 7002 raft_consensus.cc:686] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 1 LEADER]: Becoming Leader. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Running, Role: LEADER
I20260811 11:10:49.705192 7002 consensus_queue.cc:237] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.705862 6356 catalog_manager.cc:5495] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2). New cstate: current_term: 1 leader_uuid: "9f1a5e152e9640b18678e22286e06a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } health_report { overall_health: UNKNOWN } } }
I20260811 11:10:49.714356 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.711300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.29.145.2:45347)
0811 11:10:49.711342 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.711344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.711374 (+ 30us) server_negotiation.cc:407] Connection header received
0811 11:10:49.711439 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.711440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.711451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.711519 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.711672 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.712403 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.713502 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.713843 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.713914 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.714080 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.714115 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.714171 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.714249 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.714274 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:10:49.715313 6539 raft_consensus.cc:1261] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Refusing update from remote peer 9f1a5e152e9640b18678e22286e06a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.715545 7002 consensus_queue.cc:1035] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.715924 6853 raft_consensus.cc:1261] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Refusing update from remote peer 9f1a5e152e9640b18678e22286e06a7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.716336 7002 consensus_queue.cc:1035] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.720062 6356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.720163 6356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.720804 6356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Sending DeleteTablet for 3 replicas of tablet 9b8d30044d4542f6846636cc41d8b231
I20260811 11:10:49.721205 6832 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b8d30044d4542f6846636cc41d8b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:36018
I20260811 11:10:49.721300 7021 tablet_replica.cc:330] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: stopping tablet replica
I20260811 11:10:49.721359 7021 raft_consensus.cc:2229] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.721431 7021 raft_consensus.cc:2258] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.721855 6673 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b8d30044d4542f6846636cc41d8b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260811 11:10:49.721944 7022 tablet_replica.cc:330] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: stopping tablet replica
I20260811 11:10:49.721935 7021 ts_tablet_manager.cc:1905] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.721994 7022 raft_consensus.cc:2229] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:49.722060 7022 raft_consensus.cc:2258] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.722293 6515 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b8d30044d4542f6846636cc41d8b231 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:44720
I20260811 11:10:49.722409 6356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:10:49.722544 6356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:10:49.723647 7018 tablet_replica.cc:330] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: stopping tablet replica
I20260811 11:10:49.723722 7018 raft_consensus.cc:2229] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.723778 7018 raft_consensus.cc:2258] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.723874 7022 ts_tablet_manager.cc:1905] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.724558 7021 ts_tablet_manager.cc:1918] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.724622 7021 log.cc:1196] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/9b8d30044d4542f6846636cc41d8b231
I20260811 11:10:49.724864 7021 ts_tablet_manager.cc:1939] T 9b8d30044d4542f6846636cc41d8b231 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting consensus metadata
I20260811 11:10:49.725469 7022 ts_tablet_manager.cc:1918] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.725533 7022 log.cc:1196] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/9b8d30044d4542f6846636cc41d8b231
I20260811 11:10:49.725749 7022 ts_tablet_manager.cc:1939] T 9b8d30044d4542f6846636cc41d8b231 P 9f1a5e152e9640b18678e22286e06a7c: Deleting consensus metadata
I20260811 11:10:49.726141 6338 catalog_manager.cc:4841] TS 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347): tablet 9b8d30044d4542f6846636cc41d8b231 (table test-table [id=c25777820c4244428087b87c7a4f8454]) successfully deleted
I20260811 11:10:49.726589 6338 catalog_manager.cc:4841] TS 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489): tablet 9b8d30044d4542f6846636cc41d8b231 (table test-table [id=c25777820c4244428087b87c7a4f8454]) successfully deleted
I20260811 11:10:49.726907 7018 ts_tablet_manager.cc:1905] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.727141 6515 tablet_service.cc:1467] Processing CreateTablet for tablet b50210845bff403d9cdcc43d30e32eb4 (DEFAULT_TABLE table=test-table [id=d1e65c3092c74cd4bc22969f58551144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.727257 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50210845bff403d9cdcc43d30e32eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.728480 7018 ts_tablet_manager.cc:1918] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:10:49.728533 7018 log.cc:1196] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/9b8d30044d4542f6846636cc41d8b231
I20260811 11:10:49.728610 6937 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Bootstrap starting.
I20260811 11:10:49.728739 7018 ts_tablet_manager.cc:1939] T 9b8d30044d4542f6846636cc41d8b231 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting consensus metadata
I20260811 11:10:49.729072 6937 tablet_bootstrap.cc:654] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.729926 6937 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: No bootstrap required, opened a new log
I20260811 11:10:49.729970 6937 ts_tablet_manager.cc:1397] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.729959 6673 tablet_service.cc:1467] Processing CreateTablet for tablet b50210845bff403d9cdcc43d30e32eb4 (DEFAULT_TABLE table=test-table [id=d1e65c3092c74cd4bc22969f58551144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.730064 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50210845bff403d9cdcc43d30e32eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.730126 6937 raft_consensus.cc:348] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.730208 6937 raft_consensus.cc:374] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.730232 6937 raft_consensus.cc:729] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72dcf3dd0eb2471dbb15cdcf6d6f3561, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.730294 6937 consensus_queue.cc:260] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.730391 6937 ts_tablet_manager.cc:1428] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 11:10:49.731300 6940 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Bootstrap starting.
I20260811 11:10:49.731691 6940 tablet_bootstrap.cc:654] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.732492 6832 tablet_service.cc:1467] Processing CreateTablet for tablet b50210845bff403d9cdcc43d30e32eb4 (DEFAULT_TABLE table=test-table [id=d1e65c3092c74cd4bc22969f58551144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:10:49.732556 6940 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: No bootstrap required, opened a new log
I20260811 11:10:49.732599 6940 ts_tablet_manager.cc:1397] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.732610 6832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50210845bff403d9cdcc43d30e32eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:49.732741 6940 raft_consensus.cc:348] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.732803 6940 raft_consensus.cc:374] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.732822 6940 raft_consensus.cc:729] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1a5e152e9640b18678e22286e06a7c, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.732864 6940 consensus_queue.cc:260] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.732944 6940 ts_tablet_manager.cc:1428] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:49.733951 6936 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Bootstrap starting.
I20260811 11:10:49.734126 6342 catalog_manager.cc:4841] TS 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647): tablet 9b8d30044d4542f6846636cc41d8b231 (table test-table [id=c25777820c4244428087b87c7a4f8454]) successfully deleted
I20260811 11:10:49.734256 6936 tablet_bootstrap.cc:654] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:49.734881 6936 tablet_bootstrap.cc:492] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: No bootstrap required, opened a new log
I20260811 11:10:49.734937 6936 ts_tablet_manager.cc:1397] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:49.735046 6936 raft_consensus.cc:348] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.735122 6936 raft_consensus.cc:374] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:49.735144 6936 raft_consensus.cc:729] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Initialized, Role: FOLLOWER
I20260811 11:10:49.735203 6936 consensus_queue.cc:260] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.735304 6936 ts_tablet_manager.cc:1428] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 11:10:49.743765 6949 raft_consensus.cc:482] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:10:49.743870 6949 raft_consensus.cc:504] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.744043 6949 leader_election.cc:288] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347)
I20260811 11:10:49.744325 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50210845bff403d9cdcc43d30e32eb4" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c" is_pre_election: true
I20260811 11:10:49.744390 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50210845bff403d9cdcc43d30e32eb4" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" is_pre_election: true
I20260811 11:10:49.744475 6539 raft_consensus.cc:2454] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 0.
I20260811 11:10:49.744436 6693 raft_consensus.cc:2454] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 0.
I20260811 11:10:49.744626 6779 leader_election.cc:302] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 72dcf3dd0eb2471dbb15cdcf6d6f3561; no voters:
I20260811 11:10:49.744818 6949 raft_consensus.cc:2781] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:10:49.744859 6949 raft_consensus.cc:482] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:10:49.744884 6949 raft_consensus.cc:3037] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.745771 6949 raft_consensus.cc:504] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.745971 6949 leader_election.cc:288] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 election: Requested vote from peers 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647), 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347)
I20260811 11:10:49.746239 6539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50210845bff403d9cdcc43d30e32eb4" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561"
I20260811 11:10:49.746333 6539 raft_consensus.cc:3037] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.747074 6539 raft_consensus.cc:2454] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 1.
I20260811 11:10:49.747257 6779 leader_election.cc:302] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c1f2e18ce74952ac94e4cb32576fa8, 72dcf3dd0eb2471dbb15cdcf6d6f3561; no voters:
I20260811 11:10:49.747381 6693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50210845bff403d9cdcc43d30e32eb4" candidate_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1a5e152e9640b18678e22286e06a7c"
I20260811 11:10:49.747462 6693 raft_consensus.cc:3037] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:49.748212 6693 raft_consensus.cc:2454] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c1f2e18ce74952ac94e4cb32576fa8 in term 1.
I20260811 11:10:49.748426 6949 raft_consensus.cc:2781] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:49.748554 6949 raft_consensus.cc:686] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 08c1f2e18ce74952ac94e4cb32576fa8, State: Running, Role: LEADER
I20260811 11:10:49.748600 6949 consensus_queue.cc:237] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } }
I20260811 11:10:49.749212 6356 catalog_manager.cc:5495] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c1f2e18ce74952ac94e4cb32576fa8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45489 } health_report { overall_health: HEALTHY } } }
I20260811 11:10:49.758220 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.755100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.29.145.3:45489)
0811 11:10:49.755159 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.755162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.755171 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:49.755197 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.755198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.755208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.755279 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.755432 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.756222 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.757334 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.757659 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.757749 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.757925 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.757965 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.758025 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.758102 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.758135 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 11:10:49.764282 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.761042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:36549 (local address 127.29.145.2:45347)
0811 11:10:49.761092 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.761094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.761107 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:10:49.761195 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.761196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.761206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.761263 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.761435 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.762131 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.763457 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.763735 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.763814 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.764016 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.764051 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.764113 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.764186 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.764207 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:10:49.764319 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.760846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:45347 (local address 127.29.145.3:36549)
0811 11:10:49.761047 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.761062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.761134 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.761288 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.761290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.761292 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:49.761353 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.761355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.762231 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.762233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.763343 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.763347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.763837 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.763839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.763900 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.763919 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.764135 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.764137 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.764183 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.764235 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260811 11:10:49.767807 6693 raft_consensus.cc:1261] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Refusing update from remote peer 08c1f2e18ce74952ac94e4cb32576fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.768004 6539 raft_consensus.cc:1261] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Refusing update from remote peer 08c1f2e18ce74952ac94e4cb32576fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:10:49.768262 6949 consensus_queue.cc:1035] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1a5e152e9640b18678e22286e06a7c" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 45347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.768388 7049 consensus_queue.cc:1035] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72dcf3dd0eb2471dbb15cdcf6d6f3561" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:10:49.778698 6934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.775693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:55965 (local address 127.29.145.3:45489)
0811 11:10:49.775763 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:49.775765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:49.775774 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:49.775797 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:49.775798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:49.775809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:49.775868 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:10:49.776045 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.776836 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.777901 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.778219 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.778293 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.778449 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.778483 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.778535 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:49.778596 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:10:49.778617 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:10:49.778723 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:49.775478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45489 (local address 127.29.145.2:55965)
0811 11:10:49.775556 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:49.775570 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:49.775617 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:49.775898 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:49.775900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:49.775902 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:10:49.775959 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.775962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.776936 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.776937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.777789 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:49.777792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:49.778301 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:49.778302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:49.778362 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:49.778369 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:10:49.778546 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:10:49.778548 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:10:49.778593 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 11:10:49.778642 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 11:10:49.786356 6357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.786474 6357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:10:49.787227 6357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 141c9727d46a41f48d238b00664bf08f: Sending DeleteTablet for 3 replicas of tablet b50210845bff403d9cdcc43d30e32eb4
I20260811 11:10:49.787572 6673 tablet_service.cc:1514] Processing DeleteTablet for tablet b50210845bff403d9cdcc43d30e32eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46346
I20260811 11:10:49.787572 6832 tablet_service.cc:1514] Processing DeleteTablet for tablet b50210845bff403d9cdcc43d30e32eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:36018
I20260811 11:10:49.787586 6515 tablet_service.cc:1514] Processing DeleteTablet for tablet b50210845bff403d9cdcc43d30e32eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:10:49 UTC) from {username='oryx', principal='oryx/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:44720
I20260811 11:10:49.787675 7018 tablet_replica.cc:330] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: stopping tablet replica
I20260811 11:10:49.787688 7022 tablet_replica.cc:330] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: stopping tablet replica
I20260811 11:10:49.787701 7021 tablet_replica.cc:330] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: stopping tablet replica
I20260811 11:10:49.787722 7018 raft_consensus.cc:2229] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.787742 7022 raft_consensus.cc:2229] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:10:49.787756 7018 raft_consensus.cc:2258] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.787753 7021 raft_consensus.cc:2229] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:10:49.787779 7022 raft_consensus.cc:2258] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.787817 7021 raft_consensus.cc:2258] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:10:49.788211 7022 ts_tablet_manager.cc:1905] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.788211 7018 ts_tablet_manager.cc:1905] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.788254 7021 ts_tablet_manager.cc:1905] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:10:49.790099 7018 ts_tablet_manager.cc:1918] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.790172 7018 log.cc:1196] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/b50210845bff403d9cdcc43d30e32eb4
I20260811 11:10:49.790268 7021 ts_tablet_manager.cc:1918] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.790342 7021 log.cc:1196] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/b50210845bff403d9cdcc43d30e32eb4
I20260811 11:10:49.790402 7018 ts_tablet_manager.cc:1939] T b50210845bff403d9cdcc43d30e32eb4 P 72dcf3dd0eb2471dbb15cdcf6d6f3561: Deleting consensus metadata
I20260811 11:10:49.790570 7021 ts_tablet_manager.cc:1939] T b50210845bff403d9cdcc43d30e32eb4 P 08c1f2e18ce74952ac94e4cb32576fa8: Deleting consensus metadata
I20260811 11:10:49.790757 6342 catalog_manager.cc:4841] TS 72dcf3dd0eb2471dbb15cdcf6d6f3561 (127.29.145.1:37647): tablet b50210845bff403d9cdcc43d30e32eb4 (table test-table [id=d1e65c3092c74cd4bc22969f58551144]) successfully deleted
I20260811 11:10:49.790819 7022 ts_tablet_manager.cc:1918] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:10:49.790874 7022 log.cc:1196] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/b50210845bff403d9cdcc43d30e32eb4
I20260811 11:10:49.790871 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 6442
I20260811 11:10:49.791074 7022 ts_tablet_manager.cc:1939] T b50210845bff403d9cdcc43d30e32eb4 P 9f1a5e152e9640b18678e22286e06a7c: Deleting consensus metadata
I20260811 11:10:49.793450 6338 catalog_manager.cc:4841] TS 9f1a5e152e9640b18678e22286e06a7c (127.29.145.2:45347): tablet b50210845bff403d9cdcc43d30e32eb4 (table test-table [id=d1e65c3092c74cd4bc22969f58551144]) successfully deleted
I20260811 11:10:49.793519 6338 catalog_manager.cc:4841] TS 08c1f2e18ce74952ac94e4cb32576fa8 (127.29.145.3:45489): tablet b50210845bff403d9cdcc43d30e32eb4 (table test-table [id=d1e65c3092c74cd4bc22969f58551144]) successfully deleted
I20260811 11:10:49.802139 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 6601
I20260811 11:10:49.811661 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 6757
I20260811 11:10:49.819962 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 6316
2026-08-11T11:10:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3217 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[7059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[7059](info): set up 2 sockets
Aug 11 11:10:49 dist-test-slave-69rg krb5kdc[7059](info): commencing operation
krb5kdc: starting...
W20260811 11:10:50.977626 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:51 dist-test-slave-69rg krb5kdc[7059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:51.113481 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:35479
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:35479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:51.204607 7075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:51.204905 7075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:51.204936 7075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:51.206452 7075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:51.206514 7075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:51.206533 7075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:51.206550 7075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:51.206566 7075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:51.208554 7075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:35479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:35479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:51.208922 7075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:51.209270 7075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:51.216251 7082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:51.216538 7075 server_base.cc:1034] running on GCE node
W20260811 11:10:51.216250 7081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:51.216250 7084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:51.216952 7075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:51.217373 7075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:51.218590 7075 hybrid_clock.cc:648] HybridClock initialized: now 1786446651218570 us; error 53 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T11:10:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1410 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:10:51 dist-test-slave-69rg krb5kdc[7089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:10:51 dist-test-slave-69rg krb5kdc[7089](info): set up 2 sockets
Aug 11 11:10:51 dist-test-slave-69rg krb5kdc[7089](info): commencing operation
krb5kdc: starting...
W20260811 11:10:52.371812 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:10:52 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:10:52.519379 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34397
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:34397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:52.609457 7103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:52.609769 7103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:52.609815 7103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:52.611093 7103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:52.611186 7103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:52.611210 7103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:52.611233 7103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:52.611265 7103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:52.612890 7103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:34397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:52.613204 7103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:52.613459 7103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:52.618906 7111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:52.618846 7113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:52.618863 7110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:52.619379 7103 server_base.cc:1034] running on GCE node
I20260811 11:10:52.619676 7103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:52.620028 7103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:52.621241 7103 hybrid_clock.cc:648] HybridClock initialized: now 1786446652621183 us; error 40 us; skew 500 ppm
Aug 11 11:10:52 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:52.629055 7103 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:52.629674 7103 webserver.cc:460] Webserver started at http://127.29.145.62:44823/ using document root <none> and password file <none>
I20260811 11:10:52.630003 7103 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:52.630082 7103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:52.630266 7103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:52.631494 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "725c00fed457468ebe8f229cbace76f9"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "9b8a589639a4b04079daf2cd5eb3ce49"
server_key_iv: "1246ccae80c210d72d56a5357f2132f8"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.631924 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "725c00fed457468ebe8f229cbace76f9"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "9b8a589639a4b04079daf2cd5eb3ce49"
server_key_iv: "1246ccae80c210d72d56a5357f2132f8"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.633687 7103 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 11:10:52.634740 7120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:52.635026 7103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:52.635147 7103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "725c00fed457468ebe8f229cbace76f9"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "9b8a589639a4b04079daf2cd5eb3ce49"
server_key_iv: "1246ccae80c210d72d56a5357f2132f8"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.635221 7103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:52.650651 7103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:52.653251 7103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:52.653419 7103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:52.657809 7103 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:34397
I20260811 11:10:52.657912 7185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:34397 every 8 connection(s)
I20260811 11:10:52.658330 7103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:52.659346 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:10:52.662567 7186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Bootstrap starting.
I20260811 11:10:52.663280 7186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Neither blocks nor log segments found. Creating new log.
I20260811 11:10:52.663628 7186 log.cc:824] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:52.664460 7186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: No bootstrap required, opened a new log
I20260811 11:10:52.666245 7186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:52.666421 7186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:10:52.666445 7186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725c00fed457468ebe8f229cbace76f9, State: Initialized, Role: FOLLOWER
I20260811 11:10:52.666524 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7103
I20260811 11:10:52.666656 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:10:52.666621 7186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:52.666711 7186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:52.666776 7186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:52.666827 7186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:10:52.666877 30276 external_mini_cluster.cc:1428] Setting key b1a072bc138e9a6a53f0d8e77499e463
I20260811 11:10:52.667654 7186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:52.667766 7186 leader_election.cc:302] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725c00fed457468ebe8f229cbace76f9; no voters:
I20260811 11:10:52.667963 7186 leader_election.cc:288] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:10:52.667994 7191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:10:52.668171 7191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 LEADER]: Becoming Leader. State: Replica: 725c00fed457468ebe8f229cbace76f9, State: Running, Role: LEADER
I20260811 11:10:52.668284 7186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:52.668336 7191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:52.668763 7192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "725c00fed457468ebe8f229cbace76f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } } }
I20260811 11:10:52.668951 7192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:52.669656 7200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:52.669633 7193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725c00fed457468ebe8f229cbace76f9. Latest consensus state: current_term: 1 leader_uuid: "725c00fed457468ebe8f229cbace76f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } } }
I20260811 11:10:52.670162 7193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:52.670336 7200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:52.672191 7200 catalog_manager.cc:1347] Generated new cluster ID: 25fc3f6001d645c4b6a6cda498eabaf5
I20260811 11:10:52.672257 7200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:10:52 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:52.680209 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:52.667710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.29.145.62:34397)
0811 11:10:52.667961 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:52.667969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:52.667997 (+ 28us) server_negotiation.cc:407] Connection header received
0811 11:10:52.668152 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:52.668160 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:52.668217 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:52.669203 (+ 986us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:52.669352 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.670928 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.672041 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.672679 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.672784 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:52.675743 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:52.675748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:52.675748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:52.675762 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:52.678230 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:52.678772 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:52.678777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:52.678777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:52.678918 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:52.679161 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:52.679162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:52.679162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:52.679527 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:52.679696 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:52.679818 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 11:10:52.679973 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":128,"threads_started":1}
I20260811 11:10:52.680802 7200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:10:52.681406 7200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:52.686450 7200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Generated new TSK 0
I20260811 11:10:52.686661 7200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:10:52.722314 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:52.821235 7214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:52.821493 7214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:52.821535 7214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:52.822788 7214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:52.822857 7214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:52.822908 7214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:52.824400 7214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:52.824698 7214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:52.824939 7214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:52.830431 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:52.830443 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:52.830451 7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:52.830479 7214 server_base.cc:1034] running on GCE node
I20260811 11:10:52.830899 7214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:52.831152 7214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:52.832326 7214 hybrid_clock.cc:648] HybridClock initialized: now 1786446652832312 us; error 29 us; skew 500 ppm
Aug 11 11:10:52 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:52.835163 7214 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:10:52.835621 7214 webserver.cc:460] Webserver started at http://127.29.145.1:39569/ using document root <none> and password file <none>
I20260811 11:10:52.835871 7214 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:52.835942 7214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:52.836102 7214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:52.837251 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "5fda0433418f4adabe1c7b4ca854e1a2"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "6bb391da0e044ee8458c99ad3e6b4b2d"
server_key_iv: "dfa2a847700927d0b3ba49a5413d5ba0"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.837631 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "5fda0433418f4adabe1c7b4ca854e1a2"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "6bb391da0e044ee8458c99ad3e6b4b2d"
server_key_iv: "dfa2a847700927d0b3ba49a5413d5ba0"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.839673 7214 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 11:10:52.840615 7230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:52.840883 7214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:52.841027 7214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "5fda0433418f4adabe1c7b4ca854e1a2"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "6bb391da0e044ee8458c99ad3e6b4b2d"
server_key_iv: "dfa2a847700927d0b3ba49a5413d5ba0"
server_key_version: "encryptionkey@0"
I20260811 11:10:52.841125 7214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:52.848862 7214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:52.851949 7214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:52.852160 7214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:52.852456 7214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:52.852809 7214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:52.852867 7214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:52.852908 7214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:52.852938 7214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:52.859356 7214 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:42285
I20260811 11:10:52.859404 7360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:42285 every 8 connection(s)
I20260811 11:10:52.859846 7214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:10:52 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446652, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:52.869745 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7214
I20260811 11:10:52.869928 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:10:52.870193 30276 external_mini_cluster.cc:1428] Setting key 4199bbf0242e64c26fa6b38714416107
I20260811 11:10:52.871804 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:52.860820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:33689 (local address 127.29.145.62:34397)
0811 11:10:52.860894 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:52.860896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:52.861303 (+ 407us) server_negotiation.cc:407] Connection header received
0811 11:10:52.862272 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:52.862274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:52.862294 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:52.862348 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:52.863421 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.863963 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.865035 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.865285 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.865376 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:52.868419 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:52.868420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:52.868421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:52.868430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:52.870222 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:52.870686 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:52.870687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:52.870687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:52.870755 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:52.871065 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:52.871066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:52.871067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:52.871213 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:52.871291 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:52.871605 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 11:10:52.871707 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:10:52.872068 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:52.860789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:33689)
0811 11:10:52.861006 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:52.861048 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:52.862050 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:52.862439 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:52.862443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:52.863073 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:52.863300 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:52.863305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.864084 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.864086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:52.864928 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:52.864930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:52.865386 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:52.865387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:52.865529 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:52.866320 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:52.866331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:52.868252 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:52.870305 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:52.870316 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:52.870319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:52.870580 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:52.870827 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:52.870828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:52.870829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:52.870985 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:52.871294 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:52.871302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:52.871466 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 11:10:52.871582 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:10:52.871772 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260811 11:10:52.873036 7361 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34397
I20260811 11:10:52.873178 7361 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:52.873407 7361 heartbeater.cc:511] Master 127.29.145.62:34397 requested a full tablet report, sending...
I20260811 11:10:52.874083 7140 ts_manager.cc:194] Registered new tserver with Master: 5fda0433418f4adabe1c7b4ca854e1a2 (127.29.145.1:42285)
I20260811 11:10:52.875017 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:33689
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:10:52.907094 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:52.995730 7366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:52.995957 7366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:52.995982 7366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:52.997198 7366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:52.997246 7366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:52.997275 7366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:52.998889 7366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:52.999183 7366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:52.999434 7366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.005939 7372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.006007 7371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.006122 7375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.005972 7366 server_base.cc:1034] running on GCE node
I20260811 11:10:53.006417 7366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.006701 7366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.007911 7366 hybrid_clock.cc:648] HybridClock initialized: now 1786446653007859 us; error 76 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.011178 7366 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:10:53.011750 7366 webserver.cc:460] Webserver started at http://127.29.145.2:39015/ using document root <none> and password file <none>
I20260811 11:10:53.012038 7366 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.012092 7366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.012243 7366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:53.013566 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "28176a04831d487a9e4250fbb4964ac8"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "fd4e8b76e4398dd1b28e1a521fb292a6"
server_key_iv: "4cb3dbd834f397d96bfafb77c2f5c07c"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.014149 7366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "28176a04831d487a9e4250fbb4964ac8"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "fd4e8b76e4398dd1b28e1a521fb292a6"
server_key_iv: "4cb3dbd834f397d96bfafb77c2f5c07c"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.015857 7366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:53.016752 7383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.017035 7366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:53.017148 7366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "28176a04831d487a9e4250fbb4964ac8"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "fd4e8b76e4398dd1b28e1a521fb292a6"
server_key_iv: "4cb3dbd834f397d96bfafb77c2f5c07c"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.017232 7366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.025799 7366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.028234 7366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.028385 7366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.028667 7366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:53.029016 7366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:53.029050 7366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.029074 7366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:53.029088 7366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.035808 7366 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36567
I20260811 11:10:53.035849 7518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36567 every 8 connection(s)
I20260811 11:10:53.036214 7366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:53.044443 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7366
I20260811 11:10:53.044564 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:10:53.044771 30276 external_mini_cluster.cc:1428] Setting key d764a15cce13a7fb98a430783598b88c
I20260811 11:10:53.046772 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.037021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:49315 (local address 127.29.145.62:34397)
0811 11:10:53.037128 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.037131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.037298 (+ 167us) server_negotiation.cc:407] Connection header received
0811 11:10:53.037899 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.037901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.037913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.037973 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.039090 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.039602 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.040649 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.040864 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.040925 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.043322 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:53.043323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:53.043323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:53.043330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:53.045039 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:53.045569 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:53.045571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:53.045571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:53.045636 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:53.046052 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:53.046053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:53.046053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:53.046175 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:53.046264 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:53.046612 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 11:10:53.046701 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:10:53.047079 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.036964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:49315)
0811 11:10:53.037163 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.037201 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.037695 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.038062 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.038066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.038750 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.038955 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.038983 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.039708 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.039710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.040561 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.040564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.040964 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.040965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.041090 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.041694 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.041704 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.043208 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:53.045183 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:53.045190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:53.045192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:53.045488 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:53.045750 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:53.045751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:53.045752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:53.045959 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:53.046274 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:53.046281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:53.046468 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 11:10:53.046590 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 11:10:53.046784 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260811 11:10:53.047940 7519 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34397
I20260811 11:10:53.048065 7519 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:53.048254 7519 heartbeater.cc:511] Master 127.29.145.62:34397 requested a full tablet report, sending...
I20260811 11:10:53.048699 7140 ts_manager.cc:194] Registered new tserver with Master: 28176a04831d487a9e4250fbb4964ac8 (127.29.145.2:36567)
I20260811 11:10:53.049233 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:49315
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:10:53.081076 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:53.170529 7524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:53.170786 7524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:53.170826 7524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:53.172068 7524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:53.172148 7524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:53.172197 7524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:53.173746 7524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:53.174078 7524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:53.174337 7524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.180292 7530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.180384 7524 server_base.cc:1034] running on GCE node
W20260811 11:10:53.180320 7533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.180516 7531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.180927 7524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.181155 7524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.182308 7524 hybrid_clock.cc:648] HybridClock initialized: now 1786446653182286 us; error 37 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.185024 7524 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:10:53.185564 7524 webserver.cc:460] Webserver started at http://127.29.145.3:40943/ using document root <none> and password file <none>
I20260811 11:10:53.185912 7524 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.185994 7524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.186173 7524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:10:53.187434 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "c2078d36f0b949ad8dc27a31b5510830"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "87f9807da5635590468958d5f7d8dda3"
server_key_iv: "3d0ed2646e04e8bfc9138f4bb9efc758"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.187856 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "c2078d36f0b949ad8dc27a31b5510830"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "87f9807da5635590468958d5f7d8dda3"
server_key_iv: "3d0ed2646e04e8bfc9138f4bb9efc758"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.189380 7524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:10:53.190354 7542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.190685 7524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:10:53.190820 7524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "c2078d36f0b949ad8dc27a31b5510830"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "87f9807da5635590468958d5f7d8dda3"
server_key_iv: "3d0ed2646e04e8bfc9138f4bb9efc758"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.190912 7524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.201419 7524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.204028 7524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.204236 7524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.204491 7524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:53.204859 7524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:53.204923 7524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.204967 7524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:53.205008 7524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.211398 7524 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:37259
I20260811 11:10:53.211445 7677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:37259 every 8 connection(s)
I20260811 11:10:53.211781 7524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:53.217581 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7524
I20260811 11:10:53.217696 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:10:53.217916 30276 external_mini_cluster.cc:1428] Setting key add3aa578f497fba6ca372ffddf2f789
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:10:53.222636 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.212658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:32821 (local address 127.29.145.62:34397)
0811 11:10:53.212751 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.212753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.212928 (+ 175us) server_negotiation.cc:407] Connection header received
0811 11:10:53.213404 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.213405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.213417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.213481 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.214613 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.215209 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.216483 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.216685 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.216755 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.219305 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:53.219307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:53.219307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:53.219315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:53.220985 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:53.221540 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:53.221541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:53.221542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:53.221604 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:53.222043 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:53.222044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:53.222044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:53.222164 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:53.222225 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:53.222493 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 11:10:53.222566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:10:53.222834 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.212563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:32821)
0811 11:10:53.212764 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.212808 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.213304 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.213537 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.213540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.214207 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.214511 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.214517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.215303 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.215304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.216334 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.216338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.216779 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.216780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.216892 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.217481 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.217493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.219200 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:10:53.221071 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:53.221098 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:53.221101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:53.221418 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:53.221715 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:10:53.221716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:10:53.221716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:10:53.221990 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:10:53.222231 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:10:53.222236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:10:53.222362 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 11:10:53.222447 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:10:53.222579 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260811 11:10:53.223320 7678 heartbeater.cc:347] Connected to a master server at 127.29.145.62:34397
I20260811 11:10:53.223413 7678 heartbeater.cc:464] Registering TS with master...
I20260811 11:10:53.223609 7678 heartbeater.cc:511] Master 127.29.145.62:34397 requested a full tablet report, sending...
I20260811 11:10:53.224102 7140 ts_manager.cc:194] Registered new tserver with Master: c2078d36f0b949ad8dc27a31b5510830 (127.29.145.3:37259)
I20260811 11:10:53.224594 7140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:32821
I20260811 11:10:53.230450 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
Entry for principal oryx/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab.
WARNING: no policy specified for oryx/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
Entry for principal oryx/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab.
WARNING: no policy specified for oryx/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
Entry for principal oryx/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab.
I20260811 11:10:53.351727 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7214
I20260811 11:10:53.359447 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7366
I20260811 11:10:53.367938 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7524
I20260811 11:10:53.374791 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7103
I20260811 11:10:53.386219 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34397
--webserver_interface=127.29.145.62
--webserver_port=44823
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:34397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:10:53.482977 7687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:53.483279 7687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:53.483322 7687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:53.484787 7687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:10:53.484871 7687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:53.484899 7687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:53.484931 7687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:10:53.484956 7687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:10:53.486640 7687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:34397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=44823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:53.486940 7687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:53.487196 7687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.492875 7695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.493117 7687 server_base.cc:1034] running on GCE node
W20260811 11:10:53.492889 7698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.492862 7696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.493516 7687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.493872 7687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.495085 7687 hybrid_clock.cc:648] HybridClock initialized: now 1786446653495013 us; error 84 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.498082 7687 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:10:53.498654 7687 webserver.cc:460] Webserver started at http://127.29.145.62:44823/ using document root <none> and password file <none>
I20260811 11:10:53.498906 7687 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.498994 7687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.500532 7687 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:53.501362 7707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.501590 7687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:53.501734 7687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "725c00fed457468ebe8f229cbace76f9"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "9b8a589639a4b04079daf2cd5eb3ce49"
server_key_iv: "1246ccae80c210d72d56a5357f2132f8"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.502183 7687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.523624 7687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.526435 7687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.526639 7687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.531260 7687 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:34397
I20260811 11:10:53.531322 7769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:34397 every 8 connection(s)
I20260811 11:10:53.531896 7687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:10:53.534108 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7687
I20260811 11:10:53.534241 7770 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:10:53.534507 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:42285
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:34397
--webserver_port=39569
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:10:53.535463 7770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Bootstrap starting.
I20260811 11:10:53.547683 7770 log.cc:824] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:10:53.550194 7770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:10:53.550601 7770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Bootstrap complete.
I20260811 11:10:53.553581 7770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:53.553953 7770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725c00fed457468ebe8f229cbace76f9, State: Initialized, Role: FOLLOWER
I20260811 11:10:53.554171 7770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:53.554294 7770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:10:53.554361 7770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:10:53.554423 7770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:10:53.555660 7770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:53.555809 7770 leader_election.cc:302] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725c00fed457468ebe8f229cbace76f9; no voters:
I20260811 11:10:53.556057 7770 leader_election.cc:288] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:10:53.556113 7779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:10:53.556298 7779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [term 2 LEADER]: Becoming Leader. State: Replica: 725c00fed457468ebe8f229cbace76f9, State: Running, Role: LEADER
I20260811 11:10:53.556408 7770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:10:53.556488 7779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } }
I20260811 11:10:53.556911 7782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "725c00fed457468ebe8f229cbace76f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } } }
I20260811 11:10:53.557010 7782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:53.557255 7785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725c00fed457468ebe8f229cbace76f9. Latest consensus state: current_term: 2 leader_uuid: "725c00fed457468ebe8f229cbace76f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725c00fed457468ebe8f229cbace76f9" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34397 } } }
I20260811 11:10:53.557338 7785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:10:53.557538 7790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:10:53.558249 7790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:10:53.558404 7790 catalog_manager.cc:1259] Loaded cluster ID: 25fc3f6001d645c4b6a6cda498eabaf5
I20260811 11:10:53.558521 7790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:10:53.560174 7790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:10:53.560389 7790 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 725c00fed457468ebe8f229cbace76f9: Loaded TSK: 0
I20260811 11:10:53.560772 7790 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:10:53.632836 7776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:53.633111 7776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:53.633157 7776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:53.634442 7776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:53.634521 7776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:53.634570 7776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:10:53.636262 7776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.1.keytab
--principal=oryx/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:42285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=39569
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:53.636548 7776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:53.636803 7776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.643440 7813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.643553 7810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.643543 7811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.644176 7776 server_base.cc:1034] running on GCE node
I20260811 11:10:53.644433 7776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.644762 7776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.645907 7776 hybrid_clock.cc:648] HybridClock initialized: now 1786446653645894 us; error 32 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.649212 7776 init.cc:377] Logged in from keytab as oryx/127.29.145.1@KRBTEST.COM (short username oryx)
I20260811 11:10:53.649832 7776 webserver.cc:460] Webserver started at http://127.29.145.1:39569/ using document root <none> and password file <none>
I20260811 11:10:53.650110 7776 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.650195 7776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.652172 7776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:10:53.652952 7821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.653172 7776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:53.653295 7776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "5fda0433418f4adabe1c7b4ca854e1a2"
format_stamp: "Formatted at 2026-08-11 11:10:52 on dist-test-slave-69rg"
server_key: "6bb391da0e044ee8458c99ad3e6b4b2d"
server_key_iv: "dfa2a847700927d0b3ba49a5413d5ba0"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.653710 7776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.675464 7776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.678067 7776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.678265 7776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.678558 7776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:53.679095 7776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:53.679193 7776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.679247 7776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:53.679533 7776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.689945 7776 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:42285
I20260811 11:10:53.690037 7954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:42285 every 8 connection(s)
I20260811 11:10:53.690440 7776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:10:53.692330 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7776
I20260811 11:10:53.692641 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:36567
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:34397
--webserver_port=39015
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.702479 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.691342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:37769)
0811 11:10:53.691817 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.691868 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.692606 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.693444 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.693450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.694365 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.694672 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.694680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.696336 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.696338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.697289 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.697293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.698370 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.698371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.698513 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.699137 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.699147 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.702096 (+ 2949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":241,"threads_started":1}
W20260811 11:10:53.703030 7956 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:34397 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.145.62:34397: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
W20260811 11:10:53.703266 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.691455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:37769 (local address 127.29.145.62:34397)
0811 11:10:53.691702 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.691713 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.691991 (+ 278us) server_negotiation.cc:407] Connection header received
0811 11:10:53.693115 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.693122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.693187 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.693344 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.694815 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.696215 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.697406 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.698168 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.698377 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.702276 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:37769: BlockingRecv error: recv got EOF from 127.29.145.1:37769 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":117,"threads_started":1}
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:10:53.708498 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.703472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48261 (local address 127.29.145.62:34397)
0811 11:10:53.703700 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.703702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.703743 (+ 41us) server_negotiation.cc:407] Connection header received
0811 11:10:53.703863 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.703864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.703877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.703930 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.704219 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.704775 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.705496 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.705711 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.705789 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.708446 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:48261: BlockingRecv error: recv got EOF from 127.29.145.1:48261 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":112,"threads_started":1}
I20260811 11:10:53.708698 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.703371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:48261)
0811 11:10:53.703677 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.703691 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.703771 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.703932 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.703933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.704084 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.704136 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.704138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.704884 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.704885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.705402 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.705404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.705787 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.705788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.705986 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.706478 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.706483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.708638 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286}
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.713379 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.709053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:45617)
0811 11:10:53.709141 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.709153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.709201 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.709313 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.709314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.709447 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.709476 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.709477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.710181 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.710182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.710679 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.710681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.711065 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.711066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.711179 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.711624 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.711629 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.713292 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 11:10:53.713429 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.709097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45617 (local address 127.29.145.62:34397)
0811 11:10:53.709145 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.709147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.709185 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:10:53.709251 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.709252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.709263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.709311 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.709552 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.710099 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.710766 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.710989 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.711065 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.713364 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:45617: BlockingRecv error: recv got EOF from 127.29.145.1:45617 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 11:10:53.713783 7956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:10:53.792395 7960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:53.792632 7960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:53.792659 7960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:53.793809 7960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:53.793870 7960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:53.793897 7960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:10:53.795851 7960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.2.keytab
--principal=oryx/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:36567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=39015
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:53.796169 7960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:53.796445 7960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.802850 7967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.802855 7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.802855 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.803231 7960 server_base.cc:1034] running on GCE node
I20260811 11:10:53.803493 7960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.803798 7960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.805071 7960 hybrid_clock.cc:648] HybridClock initialized: now 1786446653805052 us; error 28 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.808218 7960 init.cc:377] Logged in from keytab as oryx/127.29.145.2@KRBTEST.COM (short username oryx)
I20260811 11:10:53.808800 7960 webserver.cc:460] Webserver started at http://127.29.145.2:39015/ using document root <none> and password file <none>
I20260811 11:10:53.809062 7960 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.809157 7960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.811195 7960 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:53.812310 7980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.812697 7960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:10:53.812824 7960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "28176a04831d487a9e4250fbb4964ac8"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "fd4e8b76e4398dd1b28e1a521fb292a6"
server_key_iv: "4cb3dbd834f397d96bfafb77c2f5c07c"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.813221 7960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.832446 7960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.834944 7960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.835168 7960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.835443 7960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:53.835781 7960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:53.835840 7960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.835887 7960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:53.835953 7960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.843114 7960 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36567
I20260811 11:10:53.843154 8112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36567 every 8 connection(s)
I20260811 11:10:53.843575 7960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.850903 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 7960
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.851190 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:37259
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:34397
--webserver_port=40943
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:10:53.851881 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.844472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:41933)
0811 11:10:53.844690 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.844741 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.845280 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.845612 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.845617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.846370 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.846610 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.846615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.847329 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.847330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.848226 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.848230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.848621 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.848622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.848769 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.849408 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.849424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.851712 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
W20260811 11:10:53.851918 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.844555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41933 (local address 127.29.145.62:34397)
0811 11:10:53.844670 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.844672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.844832 (+ 160us) server_negotiation.cc:407] Connection header received
0811 11:10:53.845407 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.845408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.845424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.845486 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.846740 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.847240 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.848342 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.848534 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.848614 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.851830 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:41933: BlockingRecv error: recv got EOF from 127.29.145.2:41933 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 11:10:53.852144 8114 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:34397 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.145.62:34397: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.857514 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.852474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:56361)
0811 11:10:53.852580 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.852592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.852648 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.852789 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.852790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.852954 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.853017 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.853019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.853711 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.853712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.854258 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.854259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.854715 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.854716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.854764 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.855226 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.855231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.857452 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 11:10:53.857642 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.852514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:56361 (local address 127.29.145.62:34397)
0811 11:10:53.852560 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.852562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.852649 (+ 87us) server_negotiation.cc:407] Connection header received
0811 11:10:53.852711 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.852712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.852721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.852768 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.853121 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.853634 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.854334 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.854634 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.854710 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.857572 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:56361: BlockingRecv error: recv got EOF from 127.29.145.2:56361 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:53.863132 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.857855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:44571)
0811 11:10:53.857896 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.857909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.857958 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.858262 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.858263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.858451 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.858492 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.858494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.859207 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.859208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.860083 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.860085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.860468 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.860469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.860534 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.861026 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:53.861032 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:53.863039 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260811 11:10:53.863188 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.857985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:44571 (local address 127.29.145.62:34397)
0811 11:10:53.858023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.858026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.858035 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:10:53.858059 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.858060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.858070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.858262 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.858655 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.859121 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.860177 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.860388 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.860447 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:53.863120 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:44571: BlockingRecv error: recv got EOF from 127.29.145.2:44571 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 11:10:53.863300 8114 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:10:53.943197 8117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:10:53.943444 8117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:10:53.943470 8117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:10:53.944769 8117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:10:53.944825 8117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:10:53.944855 8117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:10:53.946831 8117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/krb5kdc/oryx_127.29.145.3.keytab
--principal=oryx/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:37259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=40943
--tserver_master_addrs=127.29.145.62:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:10:53.947163 8117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:10:53.947449 8117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:10:53.954576 8127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.954576 8125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:10:53.954584 8124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:10:53.954618 8117 server_base.cc:1034] running on GCE node
I20260811 11:10:53.955067 8117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:10:53.955329 8117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:10:53.956606 8117 hybrid_clock.cc:648] HybridClock initialized: now 1786446653956552 us; error 56 us; skew 500 ppm
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446653, etypes {rep=17 tkt=17 ses=17}, oryx/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:10:53.959959 8117 init.cc:377] Logged in from keytab as oryx/127.29.145.3@KRBTEST.COM (short username oryx)
I20260811 11:10:53.960530 8117 webserver.cc:460] Webserver started at http://127.29.145.3:40943/ using document root <none> and password file <none>
I20260811 11:10:53.960813 8117 fs_manager.cc:359] Metadata directory not provided
I20260811 11:10:53.960911 8117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:10:53.962471 8117 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:10:53.963127 8134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.963361 8117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:10:53.963474 8117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "c2078d36f0b949ad8dc27a31b5510830"
format_stamp: "Formatted at 2026-08-11 11:10:53 on dist-test-slave-69rg"
server_key: "87f9807da5635590468958d5f7d8dda3"
server_key_iv: "3d0ed2646e04e8bfc9138f4bb9efc758"
server_key_version: "encryptionkey@0"
I20260811 11:10:53.963835 8117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:10:53.984081 8117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:10:53.986562 8117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:10:53.986747 8117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:10:53.987010 8117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:10:53.987349 8117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:10:53.987406 8117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.987442 8117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:10:53.987470 8117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:10:53.993758 8117 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:37259
I20260811 11:10:53.993868 8266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:37259 every 8 connection(s)
I20260811 11:10:53.994262 8117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:10:53.998654 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8117
Aug 11 11:10:53 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:10:54.003170 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:53.995167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:38843 (local address 127.29.145.62:34397)
0811 11:10:53.995230 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.995232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.995550 (+ 318us) server_negotiation.cc:407] Connection header received
0811 11:10:53.996025 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.996026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.996038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.996097 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:53.997081 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.997567 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.999019 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.999313 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.999373 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.003104 (+ 3731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:38843: BlockingRecv error: recv got EOF from 127.29.145.3:38843 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:10:54.003271 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.995080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:38843)
0811 11:10:53.995379 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:53.995425 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:53.995934 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:53.996162 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:53.996166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:53.996783 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:53.996989 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.996993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.997697 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.997698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.998925 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:53.998928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:53.999392 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:53.999393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:53.999571 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.000141 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:54.000152 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:54.003027 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":217,"threads_started":1}
W20260811 11:10:54.003741 8267 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:34397 (0 consecutive failures): Not authorized: Failed to ping master at 127.29.145.62:34397: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:54.009743 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:54.004055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:59861)
0811 11:10:54.004114 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:54.004152 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:54.004212 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:54.004526 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:54.004527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:54.004709 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:54.004764 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.004766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.005413 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.005415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.006231 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.006235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.006694 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.006695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.006773 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.007180 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:54.007185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:54.009548 (+ 2363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:10:54.009969 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:53.999228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49746 (local address 127.29.145.62:34397)
0811 11:10:53.999312 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:53.999314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:53.999442 (+ 128us) server_negotiation.cc:407] Connection header received
0811 11:10:53.999499 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:53.999500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:53.999510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:53.999564 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:54.000407 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.000937 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.002008 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.002382 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.002475 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.005478 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:10:54.005484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:10:54.005484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:10:54.005499 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:10:54.007749 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:54.008295 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:54.008299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:54.008300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:54.008444 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:10:54.008739 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:10:54.008740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:10:54.008741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:10:54.009114 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:10:54.009465 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 11:10:54.009556 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 11:10:54.009861 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 11:10:54.010221 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:54.004216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59861 (local address 127.29.145.62:34397)
0811 11:10:54.004393 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:54.004396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:54.004404 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:54.004425 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:54.004426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:54.004436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:54.004485 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:54.004848 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.005322 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.006365 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.006589 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.006692 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.010175 (+ 3483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:59861: BlockingRecv error: recv got EOF from 127.29.145.3:59861 (error 108)
Metrics: {"mutex_wait_us":476,"server-negotiator.queue_time_us":153}
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:54.016052 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:54.010340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:55447)
0811 11:10:54.010386 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:54.010399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:54.010448 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:54.010876 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:54.010877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:54.011042 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:54.011079 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.011081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.011958 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.011959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.012526 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.012528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.013021 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.013022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.013172 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.013637 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:54.013643 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:54.015989 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260811 11:10:54.016212 8267 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:10:54.016258 7958 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:54.010454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:55447 (local address 127.29.145.62:34397)
0811 11:10:54.010566 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:54.010569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:54.010577 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:54.010597 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:54.010598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:54.010690 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:54.010809 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:54.011144 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.011808 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.012706 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.012887 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.012957 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.016196 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:55447: BlockingRecv error: recv got EOF from 127.29.145.3:55447 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:54.720546 8271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:54.714431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:43733)
0811 11:10:54.714640 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:54.714656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:54.714745 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:54.714922 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:54.714923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:54.715139 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:54.715206 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.715208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.716222 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.716223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.716907 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.716910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.717356 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.717356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.717403 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.718059 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:54.718065 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:54.720454 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
W20260811 11:10:54.720834 8272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:54.714472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:43733 (local address 127.29.145.62:34397)
0811 11:10:54.714645 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:54.714647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:54.714716 (+ 69us) server_negotiation.cc:407] Connection header received
0811 11:10:54.714818 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:54.714818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:54.714831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:54.714906 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:54.715319 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.716104 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.717027 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.717271 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.717343 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.720718 (+ 3375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:43733: BlockingRecv error: recv got EOF from 127.29.145.1:43733 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:54 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:54.869669 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:54.863804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:52681)
0811 11:10:54.864005 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:54.864032 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:54.864109 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:54.864270 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:54.864271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:54.864483 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:54.864552 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.864554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.865577 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.865578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.866224 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:54.866227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.866619 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.866620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:54.866663 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.867258 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:54.867263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:54.869528 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":130,"threads_started":1}
W20260811 11:10:54.869714 8272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:54.863913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:52681 (local address 127.29.145.62:34397)
0811 11:10:54.864010 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:54.864012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:54.864089 (+ 77us) server_negotiation.cc:407] Connection header received
0811 11:10:54.864177 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:54.864178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:54.864191 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:54.864241 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:54.864651 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.865474 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.866326 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:54.866550 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:54.866613 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:54.869634 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:52681: BlockingRecv error: recv got EOF from 127.29.145.2:52681 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:55.022401 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:55.016683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:45919)
0811 11:10:55.016917 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:55.016931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:55.017020 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:55.017188 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:55.017189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:55.017468 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:55.017538 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.017540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.018345 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.018346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.018950 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.018952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.019337 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.019338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.019380 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.019917 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:55.019922 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:55.022302 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":145,"threads_started":1}
W20260811 11:10:55.022696 8272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:55.016750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:45919 (local address 127.29.145.62:34397)
0811 11:10:55.016851 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:55.016854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:55.017019 (+ 165us) server_negotiation.cc:407] Connection header received
0811 11:10:55.017068 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:55.017069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:55.017085 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:55.017137 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:55.017637 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.018248 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.019057 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.019267 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.019326 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.022449 (+ 3123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:45919: BlockingRecv error: recv got EOF from 127.29.145.3:45919 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:55.726611 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:55.721189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:45871)
0811 11:10:55.721411 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:55.721424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:55.721499 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:55.721660 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:55.721661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:55.721903 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:55.721972 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.721974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.722703 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.722705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.723269 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.723271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.723667 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.723668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.723738 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.724297 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:55.724303 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:55.726540 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
W20260811 11:10:55.726835 8276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:55.721288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45871 (local address 127.29.145.62:34397)
0811 11:10:55.721468 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:55.721470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:55.721481 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:10:55.721584 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:55.721584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:55.721596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:55.721652 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:55.722035 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.722603 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.723350 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.723575 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.723631 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.726753 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:45871: BlockingRecv error: recv got EOF from 127.29.145.1:45871 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:55 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:55.876083 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:55.870352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:55409)
0811 11:10:55.870548 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:55.870560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:55.870639 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:55.870814 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:55.870816 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:55.871066 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:55.871138 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.871141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.872023 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.872024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.872600 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:55.872602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.872989 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.872991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:55.873051 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.873579 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:55.873584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:55.876009 (+ 2425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":123,"threads_started":1}
W20260811 11:10:55.876276 8276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:55.870451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:55409 (local address 127.29.145.62:34397)
0811 11:10:55.870535 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:55.870538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:55.870636 (+ 98us) server_negotiation.cc:407] Connection header received
0811 11:10:55.870715 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:55.870716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:55.870730 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:55.870775 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:55.871262 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.871942 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.872678 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:55.872871 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:55.872927 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:55.876132 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:55409: BlockingRecv error: recv got EOF from 127.29.145.2:55409 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:56.029357 8281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:56.023239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:56077)
0811 11:10:56.023432 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:56.023450 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:56.023518 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:56.023725 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:56.023726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:56.023967 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:56.024037 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.024039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.024877 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.024878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.025855 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.025859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.026229 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.026230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.026280 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.026829 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:56.026835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:56.029279 (+ 2444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
W20260811 11:10:56.029564 8276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:56.023420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:56077 (local address 127.29.145.62:34397)
0811 11:10:56.023496 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:56.023498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:56.023508 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:56.023594 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:56.023595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:56.023610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:56.023683 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:56.024156 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.024739 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.025938 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.026151 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.026209 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.029482 (+ 3273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:56077: BlockingRecv error: recv got EOF from 127.29.145.3:56077 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:56.732455 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:56.727375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:53709)
0811 11:10:56.727553 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:56.727575 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:56.727645 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:56.727829 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:56.727830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:56.728034 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:56.728101 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.728103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.728827 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.728828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.729397 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.729399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.729764 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.729764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.729805 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.730369 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:56.730375 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:56.732367 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
W20260811 11:10:56.732647 8283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:56.727439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:53709 (local address 127.29.145.62:34397)
0811 11:10:56.727625 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:56.727627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:56.727637 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:10:56.727724 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:56.727725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:56.727739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:56.727815 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:56.728170 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.728749 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.729493 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.729693 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.729747 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.732555 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:53709: BlockingRecv error: recv got EOF from 127.29.145.1:53709 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:56 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:56.882439 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:56.876687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:46939)
0811 11:10:56.876898 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:56.876910 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:56.876982 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:56.877125 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:56.877126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:56.877333 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:56.877402 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.877404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.878213 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.878214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.878806 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:56.878807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.879282 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.879284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:56.879359 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.879931 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:56.879936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:56.882324 (+ 2388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":137,"threads_started":1}
W20260811 11:10:56.882678 8283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:56.876773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:46939 (local address 127.29.145.62:34397)
0811 11:10:56.876873 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:56.876876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:56.876981 (+ 105us) server_negotiation.cc:407] Connection header received
0811 11:10:56.877042 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:56.877043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:56.877057 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:56.877117 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:56.877491 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.878125 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.878907 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:56.879198 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:56.879280 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:56.882499 (+ 3219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:46939: BlockingRecv error: recv got EOF from 127.29.145.2:46939 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:57.036209 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:57.030297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:54467)
0811 11:10:57.030522 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:57.030538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:57.030666 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:57.030891 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:57.030892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:57.031142 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:57.031210 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.031212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.032037 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.032039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.032696 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.032698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.033115 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.033116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.033186 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.033770 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:57.033776 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:57.036118 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":119,"threads_started":1}
W20260811 11:10:57.036471 8283 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:57.030391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:54467 (local address 127.29.145.62:34397)
0811 11:10:57.030528 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:57.030531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:57.030637 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:10:57.030765 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:57.030767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:57.030785 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:57.030874 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:57.031302 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.031923 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.032794 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.033027 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.033116 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.036296 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:54467: BlockingRecv error: recv got EOF from 127.29.145.3:54467 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:57.743680 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:57.733076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:53535)
0811 11:10:57.733283 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:57.733296 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:57.733373 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:57.733573 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:57.733574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:57.733785 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:57.733882 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.733884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.734780 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.734781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.735364 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.735367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.735820 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.735821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.735873 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.741292 (+ 5419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:57.741299 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:57.743599 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":129,"threads_started":1}
W20260811 11:10:57.743934 8288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:57.733212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:53535 (local address 127.29.145.62:34397)
0811 11:10:57.733367 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:57.733369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:57.733388 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:10:57.733424 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:57.733425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:57.733437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:57.733535 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:57.733960 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.734616 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.735459 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.735743 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.735804 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.743839 (+ 8035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:53535: BlockingRecv error: recv got EOF from 127.29.145.1:53535 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:57 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:57.888644 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:57.883074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:60341)
0811 11:10:57.883303 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:57.883317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:57.883430 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:57.883629 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:57.883631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:57.883865 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:57.883936 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.883938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.884748 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.884750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.885317 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:57.885319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.885716 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.885717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:57.885779 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.886426 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:57.886433 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:57.888514 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":120,"threads_started":1}
W20260811 11:10:57.888695 8288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:57.883193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:60341 (local address 127.29.145.62:34397)
0811 11:10:57.883281 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:57.883284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:57.883403 (+ 119us) server_negotiation.cc:407] Connection header received
0811 11:10:57.883537 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:57.883538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:57.883553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:57.883618 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:57.884028 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.884604 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.885424 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:57.885635 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:57.885718 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:57.888619 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:60341: BlockingRecv error: recv got EOF from 127.29.145.2:60341 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:58.042810 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:58.036912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:59209)
0811 11:10:58.037112 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:58.037130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:58.037213 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:58.037365 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:58.037366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:58.037588 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:58.037656 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.037658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.038500 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.038502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.039162 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.039164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.039631 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.039632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.039692 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.040365 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:58.040371 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:58.042724 (+ 2353us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
W20260811 11:10:58.042935 8288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:58.037012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59209 (local address 127.29.145.62:34397)
0811 11:10:58.037128 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:58.037130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:58.037222 (+ 92us) server_negotiation.cc:407] Connection header received
0811 11:10:58.037264 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:58.037265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:58.037278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:58.037333 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:58.037738 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.038383 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.039268 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.039499 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.039574 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.042850 (+ 3276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:59209: BlockingRecv error: recv got EOF from 127.29.145.3:59209 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:58.750195 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:58.744415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:38947)
0811 11:10:58.744613 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:58.744628 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:58.744705 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:58.744962 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:58.744963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:58.745199 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:58.745267 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.745270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.746051 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.746052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.746702 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.746703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.747114 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.747116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.747169 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.747779 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:58.747785 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:58.750118 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
W20260811 11:10:58.750355 8292 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:58.744496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:38947 (local address 127.29.145.62:34397)
0811 11:10:58.744665 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:58.744668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:58.744683 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:10:58.744793 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:58.744795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:58.744812 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:58.744924 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:58.745346 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.745952 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.746787 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.747020 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.747082 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.750246 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:38947: BlockingRecv error: recv got EOF from 127.29.145.1:38947 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:58 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:58.895251 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:58.889387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:56315)
0811 11:10:58.889680 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:58.889697 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:58.889781 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:58.889984 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:58.889986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:58.890220 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:58.890287 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.890290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.891179 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.891180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.891876 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:58.891881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.892245 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.892246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:58.892291 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.892817 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:58.892822 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:58.895171 (+ 2349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":177,"threads_started":1}
W20260811 11:10:58.895414 8292 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:58.889585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:56315 (local address 127.29.145.62:34397)
0811 11:10:58.889686 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:58.889689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:58.889779 (+ 90us) server_negotiation.cc:407] Connection header received
0811 11:10:58.889880 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:58.889885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:58.889899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:58.889961 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:58.890399 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.891077 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.891975 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:58.892169 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:58.892230 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:58.895306 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:56315: BlockingRecv error: recv got EOF from 127.29.145.2:56315 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:59.050192 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:59.043736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:45761)
0811 11:10:59.044076 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:59.044100 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:59.044198 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:59.044494 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:59.044496 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:59.044724 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:59.044793 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.044796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.045718 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.045719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.046450 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.046455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.046868 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.046870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.046928 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.047592 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:59.047597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:59.050056 (+ 2459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":252,"threads_started":1}
W20260811 11:10:59.050261 8292 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:59.044001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:45761 (local address 127.29.145.62:34397)
0811 11:10:59.044252 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:59.044254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:59.044267 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:10:59.044304 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:59.044306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:59.044324 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:59.044402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:59.044911 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.045524 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.046573 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.046783 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.046851 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.050172 (+ 3321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:45761: BlockingRecv error: recv got EOF from 127.29.145.3:45761 (error 108)
Metrics: {"server-negotiator.queue_time_us":209}
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:59.760838 8295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:59.750768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:53545)
0811 11:10:59.750932 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:59.750951 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:59.751006 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:59.751212 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:59.751213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:59.751408 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:59.751485 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.751487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.752351 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.752352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.752892 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.752893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.753272 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.753273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.753332 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.758603 (+ 5271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:59.758609 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:59.760772 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
W20260811 11:10:59.761173 8296 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:59.750921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:53545 (local address 127.29.145.62:34397)
0811 11:10:59.751070 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:59.751072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:59.751080 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:10:59.751107 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:59.751108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:59.751119 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:59.751190 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:59.751572 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.752256 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.752969 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.753197 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.753252 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.761051 (+ 7799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:53545: BlockingRecv error: recv got EOF from 127.29.145.1:53545 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:10:59 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:10:59.902374 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:10:59.895899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:56871)
0811 11:10:59.896066 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 11:10:59.896081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:10:59.896177 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:10:59.896355 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:10:59.896356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:10:59.896597 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:10:59.896685 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.896688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.897494 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.897495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.898666 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:10:59.898671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.899239 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.899241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:10:59.899354 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.899977 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:10:59.899984 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:10:59.902311 (+ 2327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
W20260811 11:10:59.902442 8296 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:10:59.895971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:56871 (local address 127.29.145.62:34397)
0811 11:10:59.896053 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 11:10:59.896055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:10:59.896152 (+ 97us) server_negotiation.cc:407] Connection header received
0811 11:10:59.896260 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:10:59.896261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:10:59.896272 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:10:59.896322 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:10:59.896819 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.897405 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.898790 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:10:59.899002 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:10:59.899057 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:10:59.902394 (+ 3337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:56871: BlockingRecv error: recv got EOF from 127.29.145.2:56871 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:00.057850 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:00.050710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:55983)
0811 11:11:00.050855 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:00.050868 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:00.050940 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:00.051107 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:00.051108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:00.051359 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:00.051437 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.051440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.052204 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.052206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.053485 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.053490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.054347 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.054348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.054400 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.055005 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:00.055010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:00.057706 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
W20260811 11:11:00.057974 8296 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:00.050790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:55983 (local address 127.29.145.62:34397)
0811 11:11:00.050844 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:00.050846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:00.050920 (+ 74us) server_negotiation.cc:407] Connection header received
0811 11:11:00.051000 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:00.051001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:00.051014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:00.051080 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:00.051562 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.052129 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.053799 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.054214 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.054287 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.057867 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:55983: BlockingRecv error: recv got EOF from 127.29.145.3:55983 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:00.767795 8301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:00.761859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:38705)
0811 11:11:00.762061 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:00.762074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:00.762269 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:00.762523 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:00.762525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:00.762807 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:00.762894 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.762897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.763723 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.763724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.764341 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.764343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.764748 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.764749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.764810 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.765394 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:00.765400 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:00.767714 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
W20260811 11:11:00.767928 8302 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:00.761929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:38705 (local address 127.29.145.62:34397)
0811 11:11:00.762132 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:00.762135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:00.762251 (+ 116us) server_negotiation.cc:407] Connection header received
0811 11:11:00.762357 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:00.762358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:00.762377 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:00.762476 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:00.762985 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.763616 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.764436 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.764665 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.764735 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.767832 (+ 3097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:38705: BlockingRecv error: recv got EOF from 127.29.145.1:38705 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":111,"threads_started":1}
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:00 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:00.909672 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:00.903021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:59173)
0811 11:11:00.903222 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:00.903233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:00.903309 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:00.903529 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:00.903532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:00.903747 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:00.903816 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.903818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.904777 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.904779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.905503 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:00.905506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.906046 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.906048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:00.906108 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.906941 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:00.906947 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:00.909556 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":125,"threads_started":1}
W20260811 11:11:00.909746 8302 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:00.903111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:59173 (local address 127.29.145.62:34397)
0811 11:11:00.903212 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:00.903214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:00.903335 (+ 121us) server_negotiation.cc:407] Connection header received
0811 11:11:00.903379 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:00.903380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:00.903398 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:00.903467 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:00.903931 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.904679 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.905607 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:00.905876 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:00.906014 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:00.909674 (+ 3660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:59173: BlockingRecv error: recv got EOF from 127.29.145.2:59173 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:01.065088 8305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:01.058525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:39291)
0811 11:11:01.058787 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:01.058800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:01.058876 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:01.059001 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:01.059002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:01.059295 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:01.059368 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.059371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.060096 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.060097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.060734 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.060736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.061224 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.061225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.061279 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.062246 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:01.062254 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:01.064983 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":163,"threads_started":1}
W20260811 11:11:01.065311 8302 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:01.058605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:39291 (local address 127.29.145.62:34397)
0811 11:11:01.058687 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:01.058690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:01.058876 (+ 186us) server_negotiation.cc:407] Connection header received
0811 11:11:01.058926 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:01.058926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:01.058942 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:01.058995 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:01.059427 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.059986 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.060900 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.061118 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.061194 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.065212 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:39291: BlockingRecv error: recv got EOF from 127.29.145.3:39291 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:01.774569 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:01.768458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:39299)
0811 11:11:01.768693 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:01.768707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:01.768774 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:01.768930 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:01.768931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:01.769167 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:01.769237 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.769239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.770133 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.770135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.770784 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.770788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.771287 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.771287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.771388 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.771980 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:01.771986 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:01.774456 (+ 2470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":114,"threads_started":1}
W20260811 11:11:01.774664 8309 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:01.768588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:39299 (local address 127.29.145.62:34397)
0811 11:11:01.768749 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:01.768752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:01.768763 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:11:01.768827 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:01.768827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:01.768840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:01.768909 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:01.769310 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.770042 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.770898 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.771133 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.771197 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.774593 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:39299: BlockingRecv error: recv got EOF from 127.29.145.1:39299 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:01 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:01.916476 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:01.910315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:44995)
0811 11:11:01.910496 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:01.910510 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:01.910612 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:01.910776 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:01.910778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:01.911029 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:01.911100 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.911102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.912175 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.912177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.912911 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:01.912916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.913291 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.913292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:01.913341 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.914065 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:01.914072 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:01.916356 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
W20260811 11:11:01.916539 8309 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:01.910402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:44995 (local address 127.29.145.62:34397)
0811 11:11:01.910495 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:01.910497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:01.910607 (+ 110us) server_negotiation.cc:407] Connection header received
0811 11:11:01.910676 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:01.910677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:01.910691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:01.910752 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:01.911175 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.912067 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.912998 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:01.913214 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:01.913273 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:01.916468 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:44995: BlockingRecv error: recv got EOF from 127.29.145.2:44995 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:02.072733 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:02.065877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:42267)
0811 11:11:02.066147 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:02.066163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:02.066302 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:02.066500 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:02.066501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:02.066780 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:02.066877 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.066880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.068069 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.068071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.068941 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.068946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.069476 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.069478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.069554 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.070281 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:02.070287 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:02.072608 (+ 2321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":119,"threads_started":1}
W20260811 11:11:02.072798 8309 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:02.065978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:42267 (local address 127.29.145.62:34397)
0811 11:11:02.066067 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:02.066070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:02.066289 (+ 219us) server_negotiation.cc:407] Connection header received
0811 11:11:02.066388 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:02.066389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:02.066409 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:02.066486 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:02.066986 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.067953 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.069034 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.069366 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.069455 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.072721 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:42267: BlockingRecv error: recv got EOF from 127.29.145.3:42267 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:02.787341 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:02.775338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:43015)
0811 11:11:02.775685 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:02.775699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:02.775789 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:02.776122 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:02.776124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:02.776328 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:02.776400 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.776402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.777397 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.777399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.778117 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.778120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.778633 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.778634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.778720 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.784843 (+ 6123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:02.784860 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:02.787263 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":120,"threads_started":1}
W20260811 11:11:02.787582 8313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:02.775498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:43015 (local address 127.29.145.62:34397)
0811 11:11:02.775669 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:02.775672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:02.775858 (+ 186us) server_negotiation.cc:407] Connection header received
0811 11:11:02.775906 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:02.775908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:02.775927 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:02.776023 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:02.776518 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.777254 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.778263 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.778501 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.778579 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.787421 (+ 8842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:43015: BlockingRecv error: recv got EOF from 127.29.145.1:43015 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:02 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:02.923123 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:02.917124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:45447)
0811 11:11:02.917378 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:02.917390 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:02.917468 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:02.917643 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:02.917644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:02.917906 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:02.917973 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.917975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.918747 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.918748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.919364 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:02.919367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.919898 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.919900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:02.919975 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.920540 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:02.920546 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:02.923030 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":151,"threads_started":1}
W20260811 11:11:02.923362 8313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:02.917268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:45447 (local address 127.29.145.62:34397)
0811 11:11:02.917373 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:02.917376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:02.917477 (+ 101us) server_negotiation.cc:407] Connection header received
0811 11:11:02.917537 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:02.917538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:02.917554 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:02.917626 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:02.918096 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.918669 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.919459 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:02.919794 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:02.919868 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:02.923188 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:45447: BlockingRecv error: recv got EOF from 127.29.145.2:45447 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:03.079850 8315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:03.073378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:46691)
0811 11:11:03.073631 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:03.073645 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:03.073740 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:03.073941 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:03.073942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:03.074186 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:03.074257 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.074259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.075040 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.075041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.075619 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.075621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.076039 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.076041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.076106 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.076827 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:03.076833 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:03.079757 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":139,"threads_started":1}
W20260811 11:11:03.080011 8313 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:03.073480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:46691 (local address 127.29.145.62:34397)
0811 11:11:03.073576 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:03.073579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:03.073722 (+ 143us) server_negotiation.cc:407] Connection header received
0811 11:11:03.073809 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:03.073835 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:03.073857 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:03.073928 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:03.074347 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.074950 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.075714 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.075943 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.076016 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.079909 (+ 3893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:46691: BlockingRecv error: recv got EOF from 127.29.145.3:46691 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:03.794570 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:03.788173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:59245)
0811 11:11:03.788427 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:03.788441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:03.788541 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:03.788759 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:03.788760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:03.788984 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:03.789058 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.789060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.790311 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.790313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.790978 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.790981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.791483 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.791485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.791552 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.792207 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:03.792213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:03.794487 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":170,"threads_started":1}
W20260811 11:11:03.794971 8317 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:03.788273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:59245 (local address 127.29.145.62:34397)
0811 11:11:03.788448 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:03.788450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:03.788531 (+ 81us) server_negotiation.cc:407] Connection header received
0811 11:11:03.788618 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:03.788619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:03.788633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:03.788716 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:03.789198 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.790052 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.791134 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.791373 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.791459 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.794888 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:59245: BlockingRecv error: recv got EOF from 127.29.145.1:59245 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:03 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:03.930015 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:03.923854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:51017)
0811 11:11:03.924076 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:03.924089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:03.924173 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:03.924426 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:03.924428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:03.924672 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:03.924743 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.924746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.925507 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.925508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.926203 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:03.926206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.926990 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.926992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:03.927100 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.927770 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:03.927776 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:03.929940 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":150,"threads_started":1}
W20260811 11:11:03.930243 8317 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:03.923978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:51017 (local address 127.29.145.62:34397)
0811 11:11:03.924085 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:03.924092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:03.924204 (+ 112us) server_negotiation.cc:407] Connection header received
0811 11:11:03.924246 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:03.924247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:03.924278 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:03.924359 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:03.924834 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.925410 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.926416 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:03.926890 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:03.926982 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:03.930137 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:51017: BlockingRecv error: recv got EOF from 127.29.145.2:51017 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:04.086653 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:04.080785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:56947)
0811 11:11:04.081057 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:04.081077 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:04.081195 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:04.081360 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:04.081361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:04.081594 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:04.081683 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.081685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.082519 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.082520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.083088 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.083089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.083537 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.083539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.083615 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.084222 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:04.084228 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:04.086572 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":153,"threads_started":1}
W20260811 11:11:04.086824 8317 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:04.081036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:56947 (local address 127.29.145.62:34397)
0811 11:11:04.081125 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:04.081127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:04.081168 (+ 41us) server_negotiation.cc:407] Connection header received
0811 11:11:04.081254 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:04.081256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:04.081272 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:04.081335 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:04.081752 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.082376 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.083218 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.083436 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.083523 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.086708 (+ 3185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:56947: BlockingRecv error: recv got EOF from 127.29.145.3:56947 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:04.800907 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:04.795236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:35745)
0811 11:11:04.795522 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:04.795535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:04.795612 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:04.795794 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:04.795795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:04.796020 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:04.796085 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.796086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.796938 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.796939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.797506 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.797507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.797937 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.797938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.797984 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.798578 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:04.798584 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:04.800818 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":144,"threads_started":1}
W20260811 11:11:04.801160 8322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:04.795313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:35745 (local address 127.29.145.62:34397)
0811 11:11:04.795522 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:04.795524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:04.795664 (+ 140us) server_negotiation.cc:407] Connection header received
0811 11:11:04.795696 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:04.795697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:04.795711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:04.795774 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:04.796188 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.796848 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.797626 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.797864 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.797935 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.800930 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:35745: BlockingRecv error: recv got EOF from 127.29.145.1:35745 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":118,"threads_started":1}
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:04 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:04.937059 8323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:04.930635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:58445)
0811 11:11:04.930856 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:04.930869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:04.930951 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:04.931126 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:04.931128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:04.931351 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:04.931428 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.931430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.932276 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.932277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.932904 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:04.932906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.933306 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.933307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:04.933366 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.934105 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:04.934111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:04.936940 (+ 2829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":135,"threads_started":1}
W20260811 11:11:04.937242 8322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:04.930696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:58445 (local address 127.29.145.62:34397)
0811 11:11:04.930773 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:04.930777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:04.930957 (+ 180us) server_negotiation.cc:407] Connection header received
0811 11:11:04.931007 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:04.931007 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:04.931022 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:04.931082 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:04.931557 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.932176 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.932988 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:04.933210 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:04.933276 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:04.937136 (+ 3860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:58445: BlockingRecv error: recv got EOF from 127.29.145.2:58445 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:05.093158 8324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:05.087443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:54535)
0811 11:11:05.087717 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:05.087731 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:05.087846 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:05.088016 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:05.088017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:05.088258 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:05.088324 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.088327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.089069 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.089070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.089640 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.089642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.090141 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.090143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.090212 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.090826 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:05.090832 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:05.093018 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":138,"threads_started":1}
W20260811 11:11:05.093205 8322 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:05.087569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:54535 (local address 127.29.145.62:34397)
0811 11:11:05.087698 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:05.087701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:05.087820 (+ 119us) server_negotiation.cc:407] Connection header received
0811 11:11:05.087909 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:05.087910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:05.087934 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:05.088005 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:05.088412 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.088978 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.089736 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.090059 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.090140 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.093124 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:54535: BlockingRecv error: recv got EOF from 127.29.145.3:54535 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:05.807655 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:05.801516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:59523)
0811 11:11:05.801738 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:05.801752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:05.801913 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:05.802105 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:05.802106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:05.802351 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:05.802426 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.802428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.803287 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.803288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.803925 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.803927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.804426 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.804427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.804495 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.805083 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:05.805089 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:05.807579 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":125,"threads_started":1}
W20260811 11:11:05.807809 8326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:05.801658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:59523 (local address 127.29.145.62:34397)
0811 11:11:05.801838 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:05.801840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:05.801851 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:11:05.801989 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:05.801990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:05.802005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:05.802090 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:05.802538 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.803184 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.804035 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.804316 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.804427 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.807676 (+ 3249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:59523: BlockingRecv error: recv got EOF from 127.29.145.1:59523 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:05 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:05.943470 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:05.937808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:41391)
0811 11:11:05.938041 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:05.938059 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:05.938180 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:05.938380 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:05.938382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:05.938631 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:05.938700 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.938702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.939485 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.939486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.940187 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:05.940189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.940641 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.940642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:05.940692 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.941271 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:05.941275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:05.943392 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
W20260811 11:11:05.943677 8326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:05.938002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41391 (local address 127.29.145.62:34397)
0811 11:11:05.938097 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:05.938099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:05.938162 (+ 63us) server_negotiation.cc:407] Connection header received
0811 11:11:05.938274 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:05.938276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:05.938292 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:05.938372 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:05.938787 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.939380 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.940282 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:05.940558 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:05.940632 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:05.943601 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:41391: BlockingRecv error: recv got EOF from 127.29.145.2:41391 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:06.100092 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:06.093921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:57569)
0811 11:11:06.094166 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:06.094180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:06.094262 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:06.094417 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:06.094418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:06.094661 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:06.094728 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.094730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.095563 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.095565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.096254 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.096257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.096674 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.096675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.096738 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.097539 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:06.097546 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:06.099998 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":137,"threads_started":1}
W20260811 11:11:06.100207 8326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:06.093986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:57569 (local address 127.29.145.62:34397)
0811 11:11:06.094072 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:06.094075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:06.094258 (+ 183us) server_negotiation.cc:407] Connection header received
0811 11:11:06.094327 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:06.094328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:06.094344 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:06.094389 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:06.094829 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.095447 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.096356 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.096583 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.096650 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.100128 (+ 3478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:57569: BlockingRecv error: recv got EOF from 127.29.145.3:57569 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:06.814141 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:06.808217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:56691)
0811 11:11:06.808438 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:06.808450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:06.808532 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:06.808684 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:06.808685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:06.808898 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:06.808980 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.808982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.809734 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.809735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.810405 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.810407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.810838 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.810840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.810891 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.811520 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:06.811526 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:06.814001 (+ 2475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":139,"threads_started":1}
W20260811 11:11:06.814222 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:06.808322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:56691 (local address 127.29.145.62:34397)
0811 11:11:06.808450 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:06.808452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:06.808531 (+ 79us) server_negotiation.cc:407] Connection header received
0811 11:11:06.808582 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:06.808583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:06.808596 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:06.808666 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:06.809062 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.809648 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.810495 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.810739 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.810801 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.814133 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:56691: BlockingRecv error: recv got EOF from 127.29.145.1:56691 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:06 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:06.950153 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:06.944082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:52433)
0811 11:11:06.944389 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:06.944403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:06.944484 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:06.944724 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:06.944725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:06.944936 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:06.945009 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.945011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.945793 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.945794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.946389 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:06.946391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.946873 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.946874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:06.946931 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.947629 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:06.947635 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:06.950042 (+ 2407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":81,"threads_started":1}
W20260811 11:11:06.950312 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:06.944335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:52433 (local address 127.29.145.62:34397)
0811 11:11:06.944462 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:06.944464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:06.944477 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:11:06.944617 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:06.944618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:06.944636 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:06.944708 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:06.945109 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.945689 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.946576 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:06.946779 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:06.946847 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:06.950194 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:52433: BlockingRecv error: recv got EOF from 127.29.145.2:52433 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:07.107517 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:07.100923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:36639)
0811 11:11:07.101311 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:07.101335 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:07.101491 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:07.101714 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:07.101715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:07.102008 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:07.102076 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.102079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.102927 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.102928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.103547 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.103549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.103960 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.103962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.104033 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.104691 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:07.104697 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:07.107405 (+ 2708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":142,"threads_started":1}
W20260811 11:11:07.107559 8332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:07.101066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:36639 (local address 127.29.145.62:34397)
0811 11:11:07.101276 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:07.101279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:07.101462 (+ 183us) server_negotiation.cc:407] Connection header received
0811 11:11:07.101597 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:07.101598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:07.101625 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:07.101701 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:07.102172 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.102788 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.103646 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.103873 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.103962 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.107476 (+ 3514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:36639: BlockingRecv error: recv got EOF from 127.29.145.3:36639 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:07.826602 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:07.814960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:35325)
0811 11:11:07.815167 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:07.815180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:07.815262 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:07.815552 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:07.815554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:07.815874 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:07.815952 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.815954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.816746 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.816747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.817373 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.817381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.817785 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.817786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.817897 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.823928 (+ 6031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:07.823937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:07.826440 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
W20260811 11:11:07.826769 8337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:07.815094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:35325 (local address 127.29.145.62:34397)
0811 11:11:07.815274 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:07.815277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:07.815324 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:11:07.815401 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:07.815401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:07.815416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:07.815486 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:07.816070 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.816642 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.817475 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.817692 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.817786 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.826600 (+ 8814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:35325: BlockingRecv error: recv got EOF from 127.29.145.1:35325 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:07 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:11:07.958109 8337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:07.951057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:50037 (local address 127.29.145.62:34397)
0811 11:11:07.951132 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:07.951134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:07.951210 (+ 76us) server_negotiation.cc:407] Connection header received
0811 11:11:07.951287 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:07.951288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:07.951302 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:07.951367 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:07.951831 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.952410 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.953322 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.953776 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.953938 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.958003 (+ 4065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:50037: BlockingRecv error: recv got EOF from 127.29.145.2:50037 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:11:07.958048 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:07.950893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:50037)
0811 11:11:07.951111 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:07.951130 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:07.951210 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:07.951429 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:07.951431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:07.951673 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:07.951743 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.951746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.952508 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.952509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.953146 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:07.953149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:07.953977 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:07.953979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:07.954038 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:07.954709 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:07.954715 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:07.957812 (+ 3097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":129,"threads_started":1}
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.3@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:08.117026 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:08.108434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.3:41801)
0811 11:11:08.108683 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:08.108704 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:08.108828 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:08.109080 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:08.109082 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:08.109446 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:08.109544 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.109546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.111126 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.111128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.111962 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.111966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.112452 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.112453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.112548 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.113378 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:08.113383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:08.116800 (+ 3417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":159,"threads_started":1}
W20260811 11:11:08.117079 8337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:08.108497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:41801 (local address 127.29.145.62:34397)
0811 11:11:08.108615 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:08.108619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:08.108829 (+ 210us) server_negotiation.cc:407] Connection header received
0811 11:11:08.108917 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:08.108918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:08.108943 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:08.109036 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:08.109786 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.111020 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.112081 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.112329 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.112412 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.116962 (+ 4550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.3:41801: BlockingRecv error: recv got EOF from 127.29.145.3:41801 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.1@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:08.837229 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:08.827650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.1:52309)
0811 11:11:08.827883 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:08.827918 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:08.828019 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:08.828390 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:08.828395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:08.828943 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:08.829051 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.829054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.830580 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.830581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.831452 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.831455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.832167 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.832175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.832255 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.833239 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:08.833245 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:08.837143 (+ 3898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":127,"threads_started":1}
W20260811 11:11:08.838205 8342 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:08.827750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:52309 (local address 127.29.145.62:34397)
0811 11:11:08.827970 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:08.827972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:08.827996 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:11:08.828143 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:08.828144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:08.828168 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:08.828279 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:08.829204 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.830432 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.831585 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.832034 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.832150 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.838110 (+ 5960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.1:52309: BlockingRecv error: recv got EOF from 127.29.145.1:52309 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:11:08 dist-test-slave-69rg krb5kdc[7089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.29.145.2@KRBTEST.COM for oryx/127.29.145.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:11:08.968784 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:08.959168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:34397 (local address 127.29.145.2:49057)
0811 11:11:08.959505 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:08.959517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:08.959819 (+ 302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:08.960172 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:08.960174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:08.960543 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:08.960652 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.960655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.961729 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.961730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.962674 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:08.962677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.963226 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.963228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:08.963343 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.964043 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:08.964059 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:08.968544 (+ 4485us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.29.145.62:34397: Server oryx/127.29.145.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":221,"threads_started":1}
W20260811 11:11:08.968902 8342 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:08.959270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:49057 (local address 127.29.145.62:34397)
0811 11:11:08.959372 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:08.959374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:08.959774 (+ 400us) server_negotiation.cc:407] Connection header received
0811 11:11:08.959957 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:08.959959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:08.959994 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:08.960113 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:08.960772 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.961592 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.962798 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:08.963092 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:08.963191 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:08.968759 (+ 5568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.29.145.2:49057: BlockingRecv error: recv got EOF from 127.29.145.2:49057 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:11:08.999518 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7776
I20260811 11:11:09.017164 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7960
I20260811 11:11:09.039707 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8117
I20260811 11:11:09.053253 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 7687
2026-08-11T11:11:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17838 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T11:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:11:09Z Disabled control of system clock
I20260811 11:11:09.114867 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42805
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 11:11:09.293753 8347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:09.294422 8347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:09.294467 8347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:09.297665 8347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:11:09.297868 8347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:09.297905 8347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:09.297924 8347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:11:09.297940 8347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:11:09.300922 8347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42805
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:09.301400 8347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:09.301873 8347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:09.312711 8355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.312984 8357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.312711 8354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:09.313335 8347 server_base.cc:1034] running on GCE node
I20260811 11:11:09.313897 8347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:09.314873 8347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:09.316128 8347 hybrid_clock.cc:648] HybridClock initialized: now 1786446669316058 us; error 98 us; skew 500 ppm
I20260811 11:11:09.319624 8347 webserver.cc:460] Webserver started at http://127.29.145.62:44505/ using document root <none> and password file <none>
I20260811 11:11:09.319926 8347 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:09.319972 8347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:09.320317 8347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:09.322701 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "890cb339d4124dbdbadf19c24b1173df"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "7c920e5c9f0bbff3302ef7971ba707c7"
server_key_iv: "ab0711ceddf6d76fc57cd0dbe228d483"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.323577 8347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "890cb339d4124dbdbadf19c24b1173df"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "7c920e5c9f0bbff3302ef7971ba707c7"
server_key_iv: "ab0711ceddf6d76fc57cd0dbe228d483"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.327214 8347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:11:09.329133 8363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.329627 8347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:11:09.329877 8347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "890cb339d4124dbdbadf19c24b1173df"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "7c920e5c9f0bbff3302ef7971ba707c7"
server_key_iv: "ab0711ceddf6d76fc57cd0dbe228d483"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.330058 8347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:09.351300 8347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:09.351892 8347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:09.352197 8347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:09.361961 8347 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42805
I20260811 11:11:09.362035 8422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42805 every 8 connection(s)
I20260811 11:11:09.362860 8347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:11:09.364283 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:09.366794 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8347
I20260811 11:11:09.367002 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:11:09.367444 30276 external_mini_cluster.cc:1428] Setting key 56b82476b52195d91a04ddbd318d2ded
I20260811 11:11:09.371827 8423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df: Bootstrap starting.
I20260811 11:11:09.373225 8423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:09.373899 8423 log.cc:824] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:09.375792 8423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df: No bootstrap required, opened a new log
I20260811 11:11:09.379331 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.368670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.29.145.62:42805)
0811 11:11:09.369108 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:09.369121 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:09.369180 (+ 59us) server_negotiation.cc:407] Connection header received
0811 11:11:09.370218 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:09.370239 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:09.370412 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:09.370686 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:09.371201 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.372757 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.374660 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.375771 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.375884 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.377521 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:09.377528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:09.377528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:09.377794 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:09.377912 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:09.377929 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:09.378200 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:09.378479 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 11:11:09.378798 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":178,"threads_started":1}
I20260811 11:11:09.380816 8423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } }
I20260811 11:11:09.381008 8423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:09.381158 8423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890cb339d4124dbdbadf19c24b1173df, State: Initialized, Role: FOLLOWER
I20260811 11:11:09.381394 8423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } }
I20260811 11:11:09.381528 8423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:11:09.381639 8423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:11:09.381774 8423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:09.382805 8423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } }
I20260811 11:11:09.382956 8423 leader_election.cc:302] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 890cb339d4124dbdbadf19c24b1173df; no voters:
I20260811 11:11:09.383275 8423 leader_election.cc:288] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:11:09.383349 8429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:11:09.383649 8429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [term 1 LEADER]: Becoming Leader. State: Replica: 890cb339d4124dbdbadf19c24b1173df, State: Running, Role: LEADER
I20260811 11:11:09.383910 8429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } }
I20260811 11:11:09.383980 8423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:11:09.384738 8431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 890cb339d4124dbdbadf19c24b1173df. Latest consensus state: current_term: 1 leader_uuid: "890cb339d4124dbdbadf19c24b1173df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } } }
I20260811 11:11:09.385035 8431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [sys.catalog]: This master's current role is: LEADER
I20260811 11:11:09.385154 8430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "890cb339d4124dbdbadf19c24b1173df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890cb339d4124dbdbadf19c24b1173df" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42805 } } }
I20260811 11:11:09.385571 8430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df [sys.catalog]: This master's current role is: LEADER
I20260811 11:11:09.386058 8439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:11:09.387240 8439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:11:09.390170 8439 catalog_manager.cc:1347] Generated new cluster ID: cea882a74c8b4b4dbcea431852ec1e37
I20260811 11:11:09.390287 8439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:11:09.415619 8439 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:11:09.417091 8439 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:11:09.443934 8439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 890cb339d4124dbdbadf19c24b1173df: Generated new TSK 0
I20260811 11:11:09.444569 8439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:11:09.452651 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--builtin_ntp_servers=127.29.145.20:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 11:11:09.624672 8453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:09.625043 8453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:09.625085 8453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:09.627010 8453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:09.627084 8453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:09.627130 8453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:11:09.629302 8453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:09.629761 8453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:09.630338 8453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:09.639160 8458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.639160 8461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.639384 8459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:09.639629 8453 server_base.cc:1034] running on GCE node
I20260811 11:11:09.640165 8453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:09.640728 8453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:09.641954 8453 hybrid_clock.cc:648] HybridClock initialized: now 1786446669641901 us; error 81 us; skew 500 ppm
I20260811 11:11:09.644161 8453 webserver.cc:460] Webserver started at http://127.29.145.1:36019/ using document root <none> and password file <none>
I20260811 11:11:09.644500 8453 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:09.644554 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:09.644832 8453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:09.646812 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "80b4388205324a2ca6f57c5f46930cf8"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "d5a1a2c62637ab7023190f6da80ca52c"
server_key_iv: "7429d2a96e0886de0f78ede0044b69b3"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.647807 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "80b4388205324a2ca6f57c5f46930cf8"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "d5a1a2c62637ab7023190f6da80ca52c"
server_key_iv: "7429d2a96e0886de0f78ede0044b69b3"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.649698 8453 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260811 11:11:09.651037 8468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.651619 8453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:11:09.651787 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "80b4388205324a2ca6f57c5f46930cf8"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "d5a1a2c62637ab7023190f6da80ca52c"
server_key_iv: "7429d2a96e0886de0f78ede0044b69b3"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.651928 8453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:09.665979 8453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:09.666477 8453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:09.666711 8453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:09.667104 8453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:09.667709 8453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:09.667814 8453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.667922 8453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:09.667968 8453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.678546 8453 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:41141
I20260811 11:11:09.678673 8598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:41141 every 8 connection(s)
I20260811 11:11:09.679252 8453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:11:09.686065 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8453
I20260811 11:11:09.686252 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:11:09.686946 30276 external_mini_cluster.cc:1428] Setting key ff8b88ec0c1d815a0933254782268f06
I20260811 11:11:09.689006 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--builtin_ntp_servers=127.29.145.20:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 11:11:09.690310 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.680646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48061 (local address 127.29.145.62:42805)
0811 11:11:09.680731 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:09.680734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:09.681116 (+ 382us) server_negotiation.cc:407] Connection header received
0811 11:11:09.682266 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:09.682268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:09.682291 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:09.682386 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:09.683841 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.685161 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.687041 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.687643 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.687823 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.689553 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:09.689556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:09.689556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:09.689614 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:09.689663 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:09.689671 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:09.689896 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:09.690058 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:11:09.690148 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:11:09.690855 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.680556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42805 (local address 127.29.145.1:48061)
0811 11:11:09.680859 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:09.680953 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:09.682036 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:09.682588 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:09.682601 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:09.682620 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:09.683572 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.683579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.685313 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.685314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.686880 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.686892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.687838 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.687840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.688078 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.689145 (+ 1067us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:09.689146 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:09.689241 (+ 95us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:09.689289 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:09.689861 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:09.689879 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:09.689885 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:11:09.690058 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:11:09.690274 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":152,"threads_started":1}
I20260811 11:11:09.692109 8599 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42805
I20260811 11:11:09.692338 8599 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:09.692693 8599 heartbeater.cc:511] Master 127.29.145.62:42805 requested a full tablet report, sending...
I20260811 11:11:09.693804 8383 ts_manager.cc:194] Registered new tserver with Master: 80b4388205324a2ca6f57c5f46930cf8 (127.29.145.1:41141)
I20260811 11:11:09.695220 8383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.1:48061
W20260811 11:11:09.791080 8602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:09.791496 8602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:09.791544 8602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:09.792979 8602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:09.793032 8602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:09.793064 8602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:11:09.794754 8602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:09.795030 8602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:09.795343 8602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:09.801574 8609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.802059 8611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.802120 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:09.802219 8602 server_base.cc:1034] running on GCE node
I20260811 11:11:09.802477 8602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:09.802757 8602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:09.803973 8602 hybrid_clock.cc:648] HybridClock initialized: now 1786446669803949 us; error 42 us; skew 500 ppm
I20260811 11:11:09.805429 8602 webserver.cc:460] Webserver started at http://127.29.145.2:40369/ using document root <none> and password file <none>
I20260811 11:11:09.805706 8602 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:09.805792 8602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:09.806010 8602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:09.807140 8602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "83df7b9fbb844f43983f2777b823f7c4"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "c2ebf18dba0f7c11e7e1111490a003b7"
server_key_iv: "58c87ca4fff91c60918e272276d21910"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.807521 8602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "83df7b9fbb844f43983f2777b823f7c4"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "c2ebf18dba0f7c11e7e1111490a003b7"
server_key_iv: "58c87ca4fff91c60918e272276d21910"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.809338 8602 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:11:09.810281 8619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.810503 8602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:11:09.810654 8602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "83df7b9fbb844f43983f2777b823f7c4"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "c2ebf18dba0f7c11e7e1111490a003b7"
server_key_iv: "58c87ca4fff91c60918e272276d21910"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.810745 8602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:09.825419 8602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:09.825775 8602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:09.825968 8602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:09.826228 8602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:09.826589 8602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:09.826658 8602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.826702 8602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:09.826735 8602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.832881 8602 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:46387
I20260811 11:11:09.832942 8752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:46387 every 8 connection(s)
I20260811 11:11:09.833328 8602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:11:09.839032 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.834208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:40809 (local address 127.29.145.62:42805)
0811 11:11:09.834261 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:09.834264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:09.834586 (+ 322us) server_negotiation.cc:407] Connection header received
0811 11:11:09.835177 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:09.835179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:09.835192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:09.835242 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:09.835940 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.836458 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.837479 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.837699 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.837783 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.838742 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:09.838743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:09.838743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:09.838749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:09.838763 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:09.838767 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:09.838816 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:09.838919 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:11:09.838947 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:11:09.839291 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.834211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42805 (local address 127.29.145.2:40809)
0811 11:11:09.834448 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:09.834487 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:09.835053 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:09.835319 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:09.835324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:09.835333 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:09.835846 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.835850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.836564 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.836565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.837384 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.837386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.837789 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.837790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.837952 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.838576 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:09.838576 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:09.838607 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:09.838631 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:09.838815 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:09.838820 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:09.838823 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:11:09.838905 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:11:09.839013 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260811 11:11:09.839812 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8602
I20260811 11:11:09.839829 8754 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42805
I20260811 11:11:09.839922 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:11:09.839929 8754 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:09.840137 8754 heartbeater.cc:511] Master 127.29.145.62:42805 requested a full tablet report, sending...
I20260811 11:11:09.840171 30276 external_mini_cluster.cc:1428] Setting key e8c1dba79025563bcdcb3b3eba8a299d
I20260811 11:11:09.840582 8383 ts_manager.cc:194] Registered new tserver with Master: 83df7b9fbb844f43983f2777b823f7c4 (127.29.145.2:46387)
I20260811 11:11:09.840987 8383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.2:40809
I20260811 11:11:09.841377 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--builtin_ntp_servers=127.29.145.20:43045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 11:11:09.932014 8757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:09.932220 8757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:09.932242 8757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:09.933312 8757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:09.933350 8757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:09.933374 8757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:11:09.934774 8757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:09.934973 8757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:09.935236 8757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:09.940407 8765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.940385 8763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:09.940429 8762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:09.940496 8757 server_base.cc:1034] running on GCE node
I20260811 11:11:09.940961 8757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:09.941236 8757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:09.942384 8757 hybrid_clock.cc:648] HybridClock initialized: now 1786446669942357 us; error 45 us; skew 500 ppm
I20260811 11:11:09.943778 8757 webserver.cc:460] Webserver started at http://127.29.145.3:45375/ using document root <none> and password file <none>
I20260811 11:11:09.944041 8757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:09.944113 8757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:09.944234 8757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:09.945263 8757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "2861fec3ec984512b4df8f355fcd3b11"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "a9514d5b362ee50cb05f82141b512881"
server_key_iv: "2d19691c98fdae33049da9a79125f0b9"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.945568 8757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "2861fec3ec984512b4df8f355fcd3b11"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "a9514d5b362ee50cb05f82141b512881"
server_key_iv: "2d19691c98fdae33049da9a79125f0b9"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.946978 8757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:11:09.947870 8772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.948083 8757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:11:09.948216 8757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "2861fec3ec984512b4df8f355fcd3b11"
format_stamp: "Formatted at 2026-08-11 11:11:09 on dist-test-slave-69rg"
server_key: "a9514d5b362ee50cb05f82141b512881"
server_key_iv: "2d19691c98fdae33049da9a79125f0b9"
server_key_version: "encryptionkey@0"
I20260811 11:11:09.948297 8757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:09.957417 8757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:09.957713 8757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:09.957904 8757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:09.958158 8757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:09.958499 8757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:09.958554 8757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.958588 8757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:09.958616 8757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:09.964294 8757 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:42433
I20260811 11:11:09.964370 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:42433 every 8 connection(s)
I20260811 11:11:09.964708 8757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:11:09.967547 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8757
I20260811 11:11:09.967643 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:11:09.967816 30276 external_mini_cluster.cc:1428] Setting key 837b67711c04cf269a75a83e317b02ab
I20260811 11:11:09.970763 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.965563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:37891 (local address 127.29.145.62:42805)
0811 11:11:09.965635 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:09.965637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:09.965975 (+ 338us) server_negotiation.cc:407] Connection header received
0811 11:11:09.966561 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:09.966563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:09.966574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:09.966649 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:09.967357 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.968068 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.969360 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.969548 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.969612 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.970544 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:09.970545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:09.970546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:09.970554 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:09.970568 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:09.970572 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:09.970627 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:09.970681 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 11:11:09.970702 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:11:09.970947 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:09.965551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42805 (local address 127.29.145.3:37891)
0811 11:11:09.965858 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:09.965898 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:09.966407 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:09.966723 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:09.966726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:09.966735 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:09.967255 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.967259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.968222 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.968223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.969259 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:09.969262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:09.969630 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:09.969630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:09.969739 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:09.970412 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:09.970413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:09.970442 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:09.970464 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:09.970627 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:09.970630 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:09.970632 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:11:09.970676 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 11:11:09.970749 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":149,"threads_started":1}
I20260811 11:11:09.971457 8906 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42805
I20260811 11:11:09.971568 8906 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:09.971755 8906 heartbeater.cc:511] Master 127.29.145.62:42805 requested a full tablet report, sending...
I20260811 11:11:09.972136 8383 ts_manager.cc:194] Registered new tserver with Master: 2861fec3ec984512b4df8f355fcd3b11 (127.29.145.3:42433)
I20260811 11:11:09.972510 8383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.3:37891
I20260811 11:11:09.979843 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:11:09.982354 8427 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:09.981605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.29.145.62:42805)
0811 11:11:09.981670 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:09.981673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:09.981794 (+ 121us) server_negotiation.cc:407] Connection header received
0811 11:11:09.981923 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:09.981923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:09.981932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:09.981984 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:09.982278 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36644: BlockingRecv error: recv got EOF from 127.0.0.1:36644 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:11:09.982424 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8453
I20260811 11:11:09.999545 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8602
I20260811 11:11:10.006835 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8757
I20260811 11:11:10.016206 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8347
2026-08-11T11:11:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (948 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T11:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:11:10Z Disabled control of system clock
I20260811 11:11:10.051092 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44271
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:44271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:11:10.162400 8917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:10.162621 8917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:10.162644 8917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:10.163785 8917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:11:10.163829 8917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:10.163842 8917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:10.163856 8917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:11:10.163868 8917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:11:10.165233 8917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:44271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:44271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:10.165495 8917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:10.165860 8917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:10.171088 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.171089 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.171096 8923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:10.171527 8917 server_base.cc:1034] running on GCE node
I20260811 11:11:10.171778 8917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:10.172065 8917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:10.173230 8917 hybrid_clock.cc:648] HybridClock initialized: now 1786446670173210 us; error 34 us; skew 500 ppm
I20260811 11:11:10.174646 8917 webserver.cc:460] Webserver started at http://127.29.145.62:44763/ using document root <none> and password file <none>
I20260811 11:11:10.174853 8917 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:10.174896 8917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:10.174989 8917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:10.176054 8917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "83477d29ced84c72b7810a575d609b54"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "d17019092d753c2c0fcf4c7401e806de"
server_key_iv: "801bd428c541c62325f17fb6117c9886"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.176388 8917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "83477d29ced84c72b7810a575d609b54"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "d17019092d753c2c0fcf4c7401e806de"
server_key_iv: "801bd428c541c62325f17fb6117c9886"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.177755 8917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:11:10.178571 8934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.178758 8917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:11:10.178860 8917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "83477d29ced84c72b7810a575d609b54"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "d17019092d753c2c0fcf4c7401e806de"
server_key_iv: "801bd428c541c62325f17fb6117c9886"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.178936 8917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:10.182622 8917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:10.182905 8917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:10.183054 8917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:10.187325 8917 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:44271
I20260811 11:11:10.187361 8994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:44271 every 8 connection(s)
I20260811 11:11:10.187723 8917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:11:10.188462 8995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:10.191334 8995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54: Bootstrap starting.
I20260811 11:11:10.191881 8995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:10.192126 8995 log.cc:824] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:10.192812 8995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54: No bootstrap required, opened a new log
I20260811 11:11:10.194562 8995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } }
I20260811 11:11:10.194720 8995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:10.194746 8995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83477d29ced84c72b7810a575d609b54, State: Initialized, Role: FOLLOWER
I20260811 11:11:10.194882 8995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } }
I20260811 11:11:10.194965 8995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:11:10.194989 8995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:11:10.195017 8995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:10.195533 8995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } }
I20260811 11:11:10.195618 8995 leader_election.cc:302] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83477d29ced84c72b7810a575d609b54; no voters:
I20260811 11:11:10.195744 8995 leader_election.cc:288] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:11:10.195833 8999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:11:10.196015 8999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [term 1 LEADER]: Becoming Leader. State: Replica: 83477d29ced84c72b7810a575d609b54, State: Running, Role: LEADER
I20260811 11:11:10.196034 8995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:11:10.196117 8999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } }
I20260811 11:11:10.196803 9000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83477d29ced84c72b7810a575d609b54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } } }
I20260811 11:11:10.196911 9000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [sys.catalog]: This master's current role is: LEADER
I20260811 11:11:10.196955 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 8917
I20260811 11:11:10.197049 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:11:10.197249 30276 external_mini_cluster.cc:1428] Setting key fb5a3323075f160625e5665e2bc22cf4
I20260811 11:11:10.197331 9010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:11:10.197402 9001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83477d29ced84c72b7810a575d609b54. Latest consensus state: current_term: 1 leader_uuid: "83477d29ced84c72b7810a575d609b54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83477d29ced84c72b7810a575d609b54" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 44271 } } }
I20260811 11:11:10.197710 9001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54 [sys.catalog]: This master's current role is: LEADER
I20260811 11:11:10.198038 9010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:11:10.199677 9010 catalog_manager.cc:1347] Generated new cluster ID: 1e79e79b0f4944719a86a79ddd2ea583
I20260811 11:11:10.199733 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.197965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.29.145.62:44271)
0811 11:11:10.198221 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:10.198236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:10.198261 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:11:10.198351 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:10.198357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:10.198390 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:10.198523 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:10.199215 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:10.199219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:10.199219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:10.199224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:10.199271 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:10.199284 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:10.199379 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:10.199458 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 11:11:10.199521 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":118,"threads_started":1}
I20260811 11:11:10.199738 9010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:11:10.204311 9010 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:11:10.204478 9010 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:11:10.219434 9010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83477d29ced84c72b7810a575d609b54: Generated new TSK 0
I20260811 11:11:10.219647 9010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:11:10.224841 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--builtin_ntp_servers=127.29.145.20:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:11:10.317370 9023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:10.317620 9023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:10.317657 9023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:10.318933 9023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:10.319001 9023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:10.319051 9023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:11:10.320489 9023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:10.320911 9023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:10.321259 9023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:10.326699 9028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.326829 9031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:10.326998 9023 server_base.cc:1034] running on GCE node
W20260811 11:11:10.326695 9029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:10.327337 9023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:10.327656 9023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:10.328863 9023 hybrid_clock.cc:648] HybridClock initialized: now 1786446670328852 us; error 33 us; skew 500 ppm
I20260811 11:11:10.330502 9023 webserver.cc:460] Webserver started at http://127.29.145.1:45339/ using document root <none> and password file <none>
I20260811 11:11:10.330781 9023 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:10.330838 9023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:10.330961 9023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:10.332324 9023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "318f566ca3b14ac9ab8af0cff7f3841a"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "f5dffd993544712ea99a4059efbd0162"
server_key_iv: "788c0d0c79a276023d34b027f8410c9d"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.332746 9023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "318f566ca3b14ac9ab8af0cff7f3841a"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "f5dffd993544712ea99a4059efbd0162"
server_key_iv: "788c0d0c79a276023d34b027f8410c9d"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.334403 9023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 11:11:10.335323 9037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.335551 9023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:11:10.335670 9023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "318f566ca3b14ac9ab8af0cff7f3841a"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "f5dffd993544712ea99a4059efbd0162"
server_key_iv: "788c0d0c79a276023d34b027f8410c9d"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.335758 9023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:10.339511 9023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:10.339809 9023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:10.339973 9023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:10.340214 9023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:10.340543 9023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:10.340577 9023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.340641 9023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:10.340677 9023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.346944 9023 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:34041
I20260811 11:11:10.347029 9165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:34041 every 8 connection(s)
I20260811 11:11:10.347390 9023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:11:10.350521 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.348233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:59625 (local address 127.29.145.62:44271)
0811 11:11:10.348296 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:10.348299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:10.348508 (+ 209us) server_negotiation.cc:407] Connection header received
0811 11:11:10.348959 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:10.348960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:10.348967 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:10.349041 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:10.350235 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:10.350236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:10.350237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:10.350241 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:10.350257 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:10.350261 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:10.350315 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:10.350435 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 11:11:10.350452 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:11:10.350714 9168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.348123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44271 (local address 127.29.145.1:59625)
0811 11:11:10.348383 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:10.348425 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:10.348870 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:10.349093 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:10.349096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:10.349104 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:10.350068 (+ 964us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:10.350068 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:10.350095 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:10.350122 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:10.350334 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:10.350339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:10.350342 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:11:10.350431 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:11:10.350493 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
I20260811 11:11:10.351363 9166 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44271
I20260811 11:11:10.351475 9166 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:10.351569 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9023
I20260811 11:11:10.351620 9166 heartbeater.cc:511] Master 127.29.145.62:44271 requested a full tablet report, sending...
I20260811 11:11:10.351691 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:11:10.351902 30276 external_mini_cluster.cc:1428] Setting key dff5d7b31f6e5b0483b06a73c5972b48
I20260811 11:11:10.352254 8954 ts_manager.cc:194] Registered new tserver with Master: 318f566ca3b14ac9ab8af0cff7f3841a (127.29.145.1:34041)
I20260811 11:11:10.353209 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--builtin_ntp_servers=127.29.145.20:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:11:10.439695 9169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:10.439966 9169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:10.439992 9169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:10.441268 9169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:10.441308 9169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:10.441336 9169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:11:10.442832 9169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:10.443084 9169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:10.443396 9169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:10.449055 9174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.449066 9175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.449081 9178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:10.449434 9169 server_base.cc:1034] running on GCE node
I20260811 11:11:10.449754 9169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:10.450048 9169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:10.451241 9169 hybrid_clock.cc:648] HybridClock initialized: now 1786446670451229 us; error 34 us; skew 500 ppm
I20260811 11:11:10.452698 9169 webserver.cc:460] Webserver started at http://127.29.145.2:32891/ using document root <none> and password file <none>
I20260811 11:11:10.452980 9169 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:10.453058 9169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:10.453219 9169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:10.454416 9169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "3f9b71998ca8452280ca49d1baeae8c0"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "4e15b9e433654b9e244815e7d89bb680"
server_key_iv: "5d077f908c08b964db285a9b14663d6f"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.454800 9169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "3f9b71998ca8452280ca49d1baeae8c0"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "4e15b9e433654b9e244815e7d89bb680"
server_key_iv: "5d077f908c08b964db285a9b14663d6f"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.456205 9169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:11:10.457129 9184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.457350 9169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:11:10.457455 9169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "3f9b71998ca8452280ca49d1baeae8c0"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "4e15b9e433654b9e244815e7d89bb680"
server_key_iv: "5d077f908c08b964db285a9b14663d6f"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.457543 9169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:10.461521 9169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:10.461884 9169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:10.462042 9169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:10.462301 9169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:10.462657 9169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:10.462723 9169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.462769 9169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:10.462798 9169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.469391 9169 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:34765
I20260811 11:11:10.469446 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:34765 every 8 connection(s)
I20260811 11:11:10.469834 9169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:11:10.473145 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.470805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:47575 (local address 127.29.145.62:44271)
0811 11:11:10.470865 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:10.470867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:10.471035 (+ 168us) server_negotiation.cc:407] Connection header received
0811 11:11:10.471571 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:10.471572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:10.471579 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:10.471644 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:10.472842 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:10.472844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:10.472844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:10.472854 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:10.472878 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:10.472883 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:10.472953 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:10.473050 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 11:11:10.473067 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 11:11:10.473403 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.470671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44271 (local address 127.29.145.2:47575)
0811 11:11:10.470890 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:10.470936 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:10.471451 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:10.471714 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:10.471718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:10.471729 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:10.472681 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:10.472681 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:10.472709 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:10.472738 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:10.472950 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:10.472958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:10.472962 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:11:10.473061 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 11:11:10.473159 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260811 11:11:10.473927 9316 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44271
I20260811 11:11:10.474025 9316 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:10.474197 9316 heartbeater.cc:511] Master 127.29.145.62:44271 requested a full tablet report, sending...
I20260811 11:11:10.474622 8953 ts_manager.cc:194] Registered new tserver with Master: 3f9b71998ca8452280ca49d1baeae8c0 (127.29.145.2:34765)
I20260811 11:11:10.479516 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9169
I20260811 11:11:10.479637 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:11:10.479856 30276 external_mini_cluster.cc:1428] Setting key 643f93ce194f61b40e623fcdf2b19caa
I20260811 11:11:10.481040 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--builtin_ntp_servers=127.29.145.20:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:11:10.566560 9319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:10.566784 9319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:10.566838 9319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:10.568089 9319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:10.568132 9319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:10.568159 9319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:11:10.569449 9319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:44271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:10.569646 9319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:10.569965 9319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:10.574965 9325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.574972 9324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:10.574982 9327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:10.575345 9319 server_base.cc:1034] running on GCE node
I20260811 11:11:10.575618 9319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:10.575914 9319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:10.577138 9319 hybrid_clock.cc:648] HybridClock initialized: now 1786446670577135 us; error 42 us; skew 500 ppm
I20260811 11:11:10.578627 9319 webserver.cc:460] Webserver started at http://127.29.145.3:43837/ using document root <none> and password file <none>
I20260811 11:11:10.578904 9319 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:10.578984 9319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:10.579147 9319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:10.580247 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "706e273540da4decb342b3ce87ec3341"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "52c7308a189026629c83971ce6f7eaba"
server_key_iv: "0314aa020d9a83827082d323df7ddcfd"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.580627 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "706e273540da4decb342b3ce87ec3341"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "52c7308a189026629c83971ce6f7eaba"
server_key_iv: "0314aa020d9a83827082d323df7ddcfd"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.582300 9319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:11:10.583452 9336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.583811 9319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:11:10.583930 9319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "706e273540da4decb342b3ce87ec3341"
format_stamp: "Formatted at 2026-08-11 11:11:10 on dist-test-slave-69rg"
server_key: "52c7308a189026629c83971ce6f7eaba"
server_key_iv: "0314aa020d9a83827082d323df7ddcfd"
server_key_version: "encryptionkey@0"
I20260811 11:11:10.584031 9319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:10.587522 9319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:10.587815 9319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:10.587970 9319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:10.588183 9319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:10.588519 9319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:10.588584 9319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.588634 9319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:10.588661 9319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:10.594415 9319 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:33595
I20260811 11:11:10.594492 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:33595 every 8 connection(s)
I20260811 11:11:10.594836 9319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:11:10.597465 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9319
I20260811 11:11:10.597560 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:11:10.597779 30276 external_mini_cluster.cc:1428] Setting key 78ed1aa032ba0c48b6a9bd36ccddc090
I20260811 11:11:10.598739 9021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.595720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:58789 (local address 127.29.145.62:44271)
0811 11:11:10.595808 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:10.595810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:10.596039 (+ 229us) server_negotiation.cc:407] Connection header received
0811 11:11:10.596597 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:10.596599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:10.596614 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:10.596697 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:10.598444 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:10.598445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:10.598446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:11:10.598451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:10.598470 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:11:10.598475 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:10.598532 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:10.598637 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 11:11:10.598655 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:11:10.598920 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:10.595697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:44271 (local address 127.29.145.3:58789)
0811 11:11:10.595917 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:10.595952 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:10.596413 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:10.596820 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:10.596825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:10.596841 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:10.598224 (+ 1383us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:11:10.598225 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:10.598304 (+ 79us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:11:10.598343 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:10.598546 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:10.598551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:10.598555 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:11:10.598627 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:11:10.598677 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260811 11:11:10.599436 9462 heartbeater.cc:347] Connected to a master server at 127.29.145.62:44271
I20260811 11:11:10.599514 9462 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:10.599625 9462 heartbeater.cc:511] Master 127.29.145.62:44271 requested a full tablet report, sending...
I20260811 11:11:10.599906 8953 ts_manager.cc:194] Registered new tserver with Master: 706e273540da4decb342b3ce87ec3341 (127.29.145.3:33595)
I20260811 11:11:10.610006 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:11:10.612102 9021 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:11:10.611524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.29.145.62:44271)
0811 11:11:10.611644 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:10.611646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:10.611752 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:11:10.611837 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:10.611838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:10.611844 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:10.611897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:10.612030 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51316: BlockingRecv error: recv got EOF from 127.0.0.1:51316 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 11:11:10.612156 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9023
I20260811 11:11:10.622265 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9169
I20260811 11:11:10.632007 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9319
I20260811 11:11:10.639696 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 8917
2026-08-11T11:11:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (621 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:11:10 dist-test-slave-69rg krb5kdc[9473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:11:10 dist-test-slave-69rg krb5kdc[9473](info): set up 2 sockets
Aug 11 11:11:10 dist-test-slave-69rg krb5kdc[9473](info): commencing operation
krb5kdc: starting...
W20260811 11:11:11.786952 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:11:11 dist-test-slave-69rg krb5kdc[9473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:11:11.920789 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:36273
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:36273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:11:12.005632 9489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:12.005923 9489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:12.005965 9489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:12.007176 9489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:11:12.007256 9489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:12.007280 9489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:12.007311 9489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:11:12.007344 9489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:11:12.008870 9489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:36273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:12.009161 9489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:12.009405 9489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:12.015184 9496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.015192 9495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.015203 9498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:12.015622 9489 server_base.cc:1034] running on GCE node
I20260811 11:11:12.015939 9489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:12.016258 9489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:12.017452 9489 hybrid_clock.cc:648] HybridClock initialized: now 1786446672017426 us; error 45 us; skew 500 ppm
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:11:12.020864 9489 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:11:12.021473 9489 webserver.cc:460] Webserver started at http://127.29.145.62:40651/ using document root <none> and password file <none>
I20260811 11:11:12.021737 9489 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:12.021812 9489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:12.022024 9489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:12.023160 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "edc43acf24a94d8aa0414625941c2b4b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "236b9127d9e8e1b52006e79c8f42aced"
server_key_iv: "a23d9d7dcac7e81d428008081e4ac605"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.023562 9489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "edc43acf24a94d8aa0414625941c2b4b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "236b9127d9e8e1b52006e79c8f42aced"
server_key_iv: "a23d9d7dcac7e81d428008081e4ac605"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.025070 9489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:11:12.025996 9505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.026175 9489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:11:12.026270 9489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "edc43acf24a94d8aa0414625941c2b4b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "236b9127d9e8e1b52006e79c8f42aced"
server_key_iv: "a23d9d7dcac7e81d428008081e4ac605"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.026360 9489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:12.043936 9489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:12.046408 9489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:12.046545 9489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:12.051463 9489 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:36273
I20260811 11:11:12.051612 9567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:36273 every 8 connection(s)
I20260811 11:11:12.051964 9489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:11:12.053009 9568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:12.055745 9568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: Bootstrap starting.
I20260811 11:11:12.056387 9568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:12.056689 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9489
I20260811 11:11:12.056772 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:11:12.056962 30276 external_mini_cluster.cc:1428] Setting key 0941bb0df3c2cb9f0a2ccdb6a56886c7
I20260811 11:11:12.056934 9568 log.cc:824] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:12.058130 9568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: No bootstrap required, opened a new log
I20260811 11:11:12.060599 9568 raft_consensus.cc:348] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } }
I20260811 11:11:12.060766 9568 raft_consensus.cc:374] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:12.060814 9568 raft_consensus.cc:729] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc43acf24a94d8aa0414625941c2b4b, State: Initialized, Role: FOLLOWER
I20260811 11:11:12.060963 9568 consensus_queue.cc:260] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } }
I20260811 11:11:12.061086 9568 raft_consensus.cc:388] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:11:12.061133 9568 raft_consensus.cc:482] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:11:12.061177 9568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:12.062096 9568 raft_consensus.cc:504] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } }
I20260811 11:11:12.062243 9568 leader_election.cc:302] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edc43acf24a94d8aa0414625941c2b4b; no voters:
I20260811 11:11:12.062435 9568 leader_election.cc:288] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:11:12.062534 9573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:11:12.062697 9573 raft_consensus.cc:686] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [term 1 LEADER]: Becoming Leader. State: Replica: edc43acf24a94d8aa0414625941c2b4b, State: Running, Role: LEADER
I20260811 11:11:12.062714 9568 sys_catalog.cc:564] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:11:12.063530 9573 consensus_queue.cc:237] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } }
I20260811 11:11:12.064218 9575 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader edc43acf24a94d8aa0414625941c2b4b. Latest consensus state: current_term: 1 leader_uuid: "edc43acf24a94d8aa0414625941c2b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } } }
I20260811 11:11:12.064340 9575 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [sys.catalog]: This master's current role is: LEADER
I20260811 11:11:12.064343 9574 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edc43acf24a94d8aa0414625941c2b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc43acf24a94d8aa0414625941c2b4b" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36273 } } }
I20260811 11:11:12.064944 9574 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b [sys.catalog]: This master's current role is: LEADER
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
W20260811 11:11:12.066234 9589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:11:12.067776 9589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:11:12.067857 9583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:11:12.068045 9583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:11:12.069790 9583 catalog_manager.cc:1347] Generated new cluster ID: 5b38663657ef4695a6b3aa5f741d3482
I20260811 11:11:12.069864 9583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:11:12.072642 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.057627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50556 (local address 127.29.145.62:36273)
0811 11:11:12.057883 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.057891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.057922 (+ 31us) server_negotiation.cc:407] Connection header received
0811 11:11:12.058494 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.058504 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.058557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.058684 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:12.059535 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.061112 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.062388 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.063201 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.063319 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.066684 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:12.066689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:12.066689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:11:12.066704 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:12.070386 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.071182 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.071186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.071186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.071297 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.071546 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.071547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.071547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.071843 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:12.071983 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.072125 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.072260 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260811 11:11:12.079077 9583 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:11:12.079595 9583 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:11:12.082288 9583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: Generated new TSK 0
I20260811 11:11:12.082482 9583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:11:12.121519 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:11:12.234627 9595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:12.234836 9595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:12.234861 9595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:12.236017 9595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:12.236064 9595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:12.236092 9595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:11:12.237641 9595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:12.237913 9595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:12.238188 9595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:11:12.243803 9595 server_base.cc:1034] running on GCE node
W20260811 11:11:12.243790 9600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.243868 9604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.243834 9602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:12.244220 9595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:12.244442 9595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:12.245652 9595 hybrid_clock.cc:648] HybridClock initialized: now 1786446672245646 us; error 46 us; skew 500 ppm
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:11:12.248594 9595 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:11:12.249112 9595 webserver.cc:460] Webserver started at http://127.29.145.1:38483/ using document root <none> and password file <none>
I20260811 11:11:12.249375 9595 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:12.249444 9595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:12.249594 9595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:12.250748 9595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "919f4925daa148aaa812ab69a0170833"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "dd27a12ceb7fc27b47dcc736fcd0b3b9"
server_key_iv: "c1414761f590433a8aa1947d74c55986"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.251125 9595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "919f4925daa148aaa812ab69a0170833"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "dd27a12ceb7fc27b47dcc736fcd0b3b9"
server_key_iv: "c1414761f590433a8aa1947d74c55986"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.252501 9595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:11:12.253396 9611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.253728 9595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:11:12.253878 9595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "919f4925daa148aaa812ab69a0170833"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "dd27a12ceb7fc27b47dcc736fcd0b3b9"
server_key_iv: "c1414761f590433a8aa1947d74c55986"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.253962 9595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:12.267220 9595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:12.269632 9595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:12.269811 9595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:12.270080 9595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:12.270418 9595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:12.270471 9595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.270506 9595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:12.270534 9595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.276654 9595 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:45305
I20260811 11:11:12.276718 9745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:45305 every 8 connection(s)
I20260811 11:11:12.277033 9595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:11:12.279513 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9595
I20260811 11:11:12.279634 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:11:12.279843 30276 external_mini_cluster.cc:1428] Setting key f70d8b06c155e8516df6ed1cd6fa9993
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:11:12.287828 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.277741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36273 (local address 127.29.145.1:60817)
0811 11:11:12.277935 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.277977 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.278529 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.278790 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.278794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.279377 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:12.279570 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.279574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.280512 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.280513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.281883 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.281887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.282334 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.282335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.282489 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.283094 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:12.283103 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:12.284476 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:12.286248 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.286256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.286259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.286587 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.286810 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.286811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.286811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.286952 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.287309 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:12.287318 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:12.287440 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.287529 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.287654 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260811 11:11:12.287830 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.277791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:60817 (local address 127.29.145.62:36273)
0811 11:11:12.277902 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.277904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.278056 (+ 152us) server_negotiation.cc:407] Connection header received
0811 11:11:12.278640 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.278641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.278654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.278709 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:12.279677 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.280405 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.281980 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.282233 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.282303 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.284578 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:12.284579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:12.284580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:11:12.284588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:12.286162 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.286668 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.286669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.286670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.286736 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.287051 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.287052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.287053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.287222 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:12.287313 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.287587 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.287709 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 11:11:12.288432 9746 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36273
I20260811 11:11:12.288552 9746 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:12.288779 9746 heartbeater.cc:511] Master 127.29.145.62:36273 requested a full tablet report, sending...
I20260811 11:11:12.289366 9526 ts_manager.cc:194] Registered new tserver with Master: 919f4925daa148aaa812ab69a0170833 (127.29.145.1:45305)
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
I20260811 11:11:12.290086 9526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:60817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:11:12.314540 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:11:12.409006 9751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:12.409211 9751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:12.409235 9751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:12.410398 9751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:12.410439 9751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:12.410466 9751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:11:12.411937 9751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:12.412194 9751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:12.412444 9751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:11:12.417934 9751 server_base.cc:1034] running on GCE node
W20260811 11:11:12.417920 9756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.417906 9757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.417915 9759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:12.418435 9751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:12.418749 9751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:12.419924 9751 hybrid_clock.cc:648] HybridClock initialized: now 1786446672419872 us; error 70 us; skew 500 ppm
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:11:12.423370 9751 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:11:12.423929 9751 webserver.cc:460] Webserver started at http://127.29.145.2:32899/ using document root <none> and password file <none>
I20260811 11:11:12.424206 9751 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:12.424286 9751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:12.424440 9751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:12.425537 9751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "4dfae5d1959c4313a7465a644a4afa49"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "863e50b9743c570b5201cf8c55e61d96"
server_key_iv: "06659c7e85e459de2d88da362ad3d7e6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.425971 9751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "4dfae5d1959c4313a7465a644a4afa49"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "863e50b9743c570b5201cf8c55e61d96"
server_key_iv: "06659c7e85e459de2d88da362ad3d7e6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.427418 9751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:11:12.428195 9767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.428427 9751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:11:12.428560 9751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "4dfae5d1959c4313a7465a644a4afa49"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "863e50b9743c570b5201cf8c55e61d96"
server_key_iv: "06659c7e85e459de2d88da362ad3d7e6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.428658 9751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:12.447644 9751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:12.449784 9751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:12.450004 9751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:12.450271 9751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:12.450641 9751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:12.450703 9751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.450747 9751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:12.450775 9751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.457085 9751 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:37971
I20260811 11:11:12.457178 9898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:37971 every 8 connection(s)
I20260811 11:11:12.457535 9751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:11:12.461406 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9751
I20260811 11:11:12.461521 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:11:12.461740 30276 external_mini_cluster.cc:1428] Setting key ac147a935e167d21782be5a67fcc37bc
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:11:12.468324 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.458503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:54663 (local address 127.29.145.62:36273)
0811 11:11:12.458576 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.458579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.458817 (+ 238us) server_negotiation.cc:407] Connection header received
0811 11:11:12.459379 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.459380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.459394 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.459455 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:12.460432 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.460965 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.462493 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.462791 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.462868 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.465410 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:12.465412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:12.465412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:11:12.465421 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:12.466896 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.467327 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.467329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.467329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.467390 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.467654 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.467655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.467655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.467779 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:12.467857 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.468144 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.468224 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:11:12.468511 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.458461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36273 (local address 127.29.145.2:54663)
0811 11:11:12.458691 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.458729 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.459260 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.459530 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.459534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.460131 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:12.460328 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.460332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.461072 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.461074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.462401 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.462404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.462888 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.462889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.463020 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.463684 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:12.463694 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:12.465302 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:12.467002 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.467010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.467012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.467246 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.467460 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.467461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.467462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.467576 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.467848 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:12.467855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:12.468000 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.468106 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.468271 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
I20260811 11:11:12.469043 9899 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36273
I20260811 11:11:12.469152 9899 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:12.469347 9899 heartbeater.cc:511] Master 127.29.145.62:36273 requested a full tablet report, sending...
I20260811 11:11:12.469794 9526 ts_manager.cc:194] Registered new tserver with Master: 4dfae5d1959c4313a7465a644a4afa49 (127.29.145.2:37971)
I20260811 11:11:12.470196 9526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:54663
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:11:12.493295 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:11:12.579802 9904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:11:12.580009 9904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:11:12.580032 9904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:11:12.581130 9904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:11:12.581166 9904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:11:12.581192 9904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:11:12.582732 9904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:11:12.582980 9904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:11:12.583256 9904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:11:12.588735 9911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.588784 9909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:11:12.589071 9913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:11:12.589305 9904 server_base.cc:1034] running on GCE node
I20260811 11:11:12.589520 9904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:11:12.589778 9904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:11:12.590965 9904 hybrid_clock.cc:648] HybridClock initialized: now 1786446672590939 us; error 40 us; skew 500 ppm
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:11:12.593562 9904 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:11:12.594043 9904 webserver.cc:460] Webserver started at http://127.29.145.3:46161/ using document root <none> and password file <none>
I20260811 11:11:12.594236 9904 fs_manager.cc:359] Metadata directory not provided
I20260811 11:11:12.594280 9904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:11:12.594372 9904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:11:12.595404 9904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "1c4ebeeabec04e3e8633c0a72572816b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "1326f5a84216dafbeca353ef9849f8e2"
server_key_iv: "ac3d7980b44c092b2d1021129d6bb0d6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.595726 9904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "1c4ebeeabec04e3e8633c0a72572816b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "1326f5a84216dafbeca353ef9849f8e2"
server_key_iv: "ac3d7980b44c092b2d1021129d6bb0d6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.597018 9904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:11:12.597749 9920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.598003 9904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:11:12.598114 9904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "1c4ebeeabec04e3e8633c0a72572816b"
format_stamp: "Formatted at 2026-08-11 11:11:12 on dist-test-slave-69rg"
server_key: "1326f5a84216dafbeca353ef9849f8e2"
server_key_iv: "ac3d7980b44c092b2d1021129d6bb0d6"
server_key_version: "encryptionkey@0"
I20260811 11:11:12.598186 9904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:11:12.612082 9904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:11:12.614169 9904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:11:12.614283 9904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:11:12.614467 9904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:11:12.614735 9904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:11:12.614768 9904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.614794 9904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:11:12.614806 9904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:11:12.620630 9904 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:35751
I20260811 11:11:12.620718 10053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:35751 every 8 connection(s)
I20260811 11:11:12.621026 9904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
Aug 11 11:11:12 dist-test-slave-69rg krb5kdc[9473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446672, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:11:12.629932 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 9904
I20260811 11:11:12.630074 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:11:12.630317 30276 external_mini_cluster.cc:1428] Setting key 390cdf82683cf0d1c68979c5b263d2c8
I20260811 11:11:12.632061 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.621793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:36681 (local address 127.29.145.62:36273)
0811 11:11:12.621890 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.621892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.622105 (+ 213us) server_negotiation.cc:407] Connection header received
0811 11:11:12.622653 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.622654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.622665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.622714 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:12.623712 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.624284 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.625731 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.626024 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.626093 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.628532 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:12.628534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:12.628534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:11:12.628541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:12.630318 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.630902 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.630903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.630903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.630957 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.631364 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.631365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.631366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.631517 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:12.631571 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.631847 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.631961 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:11:12.632230 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.621770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36273 (local address 127.29.145.3:36681)
0811 11:11:12.621972 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.622008 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.622532 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.622775 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.622779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.623358 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:11:12.623551 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.623556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.624616 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.624617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.625572 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.625576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.626126 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.626127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.626240 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.626837 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:11:12.626846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:11:12.628422 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:11:12.630407 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.630415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.630417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.630744 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.631026 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:11:12.631027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:11:12.631027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:11:12.631165 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:11:12.631574 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:11:12.631579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:11:12.631703 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.631793 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.631980 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":138,"threads_started":1}
I20260811 11:11:12.632670 10054 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36273
I20260811 11:11:12.632766 10054 heartbeater.cc:464] Registering TS with master...
I20260811 11:11:12.632921 10054 heartbeater.cc:511] Master 127.29.145.62:36273 requested a full tablet report, sending...
I20260811 11:11:12.633354 9526 ts_manager.cc:194] Registered new tserver with Master: 1c4ebeeabec04e3e8633c0a72572816b (127.29.145.3:35751)
I20260811 11:11:12.633687 9526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:36681
I20260811 11:11:12.642410 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:11:12.650022 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.643758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.29.145.62:36273)
0811 11:11:12.643829 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.643831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.643897 (+ 66us) server_negotiation.cc:407] Connection header received
0811 11:11:12.643949 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.643949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.643958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.643999 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:11:12.644572 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.645062 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.645781 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.646052 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.646112 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.647331 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:11:12.647333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:11:12.647333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:11:12.647340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:11:12.648806 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.649152 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.649153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.649154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.649215 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:11:12.649393 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:11:12.649394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:11:12.649394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:11:12.649539 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:11:12.649642 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.649750 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.649891 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:11:12.651925 9526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:11:12.653582 9526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:11:12.660069 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.29.145.1:45305)
0811 11:11:12.656414 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.656420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.656526 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:11:12.656600 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.656602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.656619 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.656695 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:11:12.656970 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.657867 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.659147 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.659650 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.659743 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.659809 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.659940 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.659981 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260811 11:11:12.660060 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:45305 (local address 127.0.0.1:40054)
0811 11:11:12.656468 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.656477 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.656529 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.656789 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.656794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.656797 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:11:12.656870 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.656872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.657966 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.657968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.659056 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.659059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.659746 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.659747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.659830 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.659844 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.659920 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.659979 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":64,"threads_started":1}
I20260811 11:11:12.660959 10070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:35751 (local address 127.0.0.1:53978)
0811 11:11:12.656237 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.656279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.656350 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.656918 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.656920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.656922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:11:12.656982 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.656984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.658560 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.658562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.659633 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.659636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.660697 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.660699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.660761 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.660766 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.660833 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.660876 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":33,"thread_start_us":141,"threads_started":1}
I20260811 11:11:12.660940 9666 tablet_service.cc:1467] Processing CreateTablet for tablet 76b020819a714eada4aff9248811b6ce (DEFAULT_TABLE table=test-table [id=6e18916ea89c4dc68528075bad3409d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:11:12.661010 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.29.145.3:35751)
0811 11:11:12.656663 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.656668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.656677 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:11:12.656713 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.656715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.656731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.656842 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:11:12.657079 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.658445 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.659791 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.660602 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.660687 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.660745 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.660849 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.660892 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":93,"threads_started":1}
I20260811 11:11:12.661388 9666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b020819a714eada4aff9248811b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:12.662356 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:37971 (local address 127.0.0.1:39236)
0811 11:11:12.656515 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.656527 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.656583 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.657073 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.657074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.657075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:11:12.657107 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.657109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.658543 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.658544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.660904 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.660908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.662109 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.662110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.662183 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.662188 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.662242 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.662285 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1110,"thread_start_us":118,"threads_started":1}
I20260811 11:11:12.662662 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.656605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.29.145.2:37971)
0811 11:11:12.656849 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.656854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.656864 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:11:12.656902 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.656904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.656918 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.656991 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:11:12.657185 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.658421 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.661014 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.661920 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.661987 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.662045 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.662524 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.662575 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260811 11:11:12.662636 9978 tablet_service.cc:1467] Processing CreateTablet for tablet 76b020819a714eada4aff9248811b6ce (DEFAULT_TABLE table=test-table [id=6e18916ea89c4dc68528075bad3409d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:11:12.663023 9978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b020819a714eada4aff9248811b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:12.664727 10076 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Bootstrap starting.
I20260811 11:11:12.665432 10076 tablet_bootstrap.cc:654] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:12.665372 9825 tablet_service.cc:1467] Processing CreateTablet for tablet 76b020819a714eada4aff9248811b6ce (DEFAULT_TABLE table=test-table [id=6e18916ea89c4dc68528075bad3409d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:11:12.665771 9825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b020819a714eada4aff9248811b6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:11:12.666042 10077 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Bootstrap starting.
I20260811 11:11:12.666208 10076 log.cc:824] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:12.666843 10077 tablet_bootstrap.cc:654] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:12.667092 10076 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: No bootstrap required, opened a new log
I20260811 11:11:12.667176 10076 ts_tablet_manager.cc:1397] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:11:12.667402 10077 log.cc:824] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:12.669106 10081 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Bootstrap starting.
I20260811 11:11:12.669342 10076 raft_consensus.cc:348] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.669528 10076 raft_consensus.cc:374] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:12.669574 10076 raft_consensus.cc:729] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919f4925daa148aaa812ab69a0170833, State: Initialized, Role: FOLLOWER
I20260811 11:11:12.669687 10077 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: No bootstrap required, opened a new log
I20260811 11:11:12.669749 10081 tablet_bootstrap.cc:654] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Neither blocks nor log segments found. Creating new log.
I20260811 11:11:12.669775 10077 ts_tablet_manager.cc:1397] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:11:12.669723 10076 consensus_queue.cc:260] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.670104 9746 heartbeater.cc:503] Master 127.29.145.62:36273 was elected leader, sending a full tablet report...
I20260811 11:11:12.670102 10076 ts_tablet_manager.cc:1428] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:11:12.670393 10081 log.cc:824] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Log is configured to *not* fsync() on all Append() calls
I20260811 11:11:12.671159 10081 tablet_bootstrap.cc:492] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: No bootstrap required, opened a new log
I20260811 11:11:12.671232 10081 ts_tablet_manager.cc:1397] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:11:12.672008 10077 raft_consensus.cc:348] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.672179 10077 raft_consensus.cc:374] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:12.672216 10077 raft_consensus.cc:729] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4ebeeabec04e3e8633c0a72572816b, State: Initialized, Role: FOLLOWER
I20260811 11:11:12.672333 10077 consensus_queue.cc:260] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.672612 10077 ts_tablet_manager.cc:1428] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:11:12.672634 10054 heartbeater.cc:503] Master 127.29.145.62:36273 was elected leader, sending a full tablet report...
I20260811 11:11:12.673301 10081 raft_consensus.cc:348] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.673470 10081 raft_consensus.cc:374] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:11:12.673523 10081 raft_consensus.cc:729] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfae5d1959c4313a7465a644a4afa49, State: Initialized, Role: FOLLOWER
I20260811 11:11:12.673645 10081 consensus_queue.cc:260] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.673949 10081 ts_tablet_manager.cc:1428] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:11:12.673982 9899 heartbeater.cc:503] Master 127.29.145.62:36273 was elected leader, sending a full tablet report...
W20260811 11:11:12.708230 9900 tablet.cc:2369] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:11:12.777871 9747 tablet.cc:2369] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:11:12.871649 10055 tablet.cc:2369] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:11:12.948670 10085 raft_consensus.cc:482] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:11:12.948819 10085 raft_consensus.cc:504] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.949250 10085 leader_election.cc:288] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c4ebeeabec04e3e8633c0a72572816b (127.29.145.3:35751), 919f4925daa148aaa812ab69a0170833 (127.29.145.1:45305)
I20260811 11:11:12.952165 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.949571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:36125 (local address 127.29.145.3:35751)
0811 11:11:12.949622 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.949624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.949633 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:11:12.949664 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.949665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.949676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.949725 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:11:12.949913 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.950404 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.951428 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.951805 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.951890 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.951899 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.952054 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.952093 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:11:12.952165 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.949450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:35751 (local address 127.29.145.2:36125)
0811 11:11:12.949493 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.949506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.949561 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.949741 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.949742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.949743 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:11:12.949832 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.949834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.950491 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.950493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.951322 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.951325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.951888 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.951889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.951973 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.951976 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.952047 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.952090 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 11:11:12.952433 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.949511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:45305 (local address 127.29.145.2:45521)
0811 11:11:12.949719 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:12.949733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:11:12.949867 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:11:12.950052 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:11:12.950053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:11:12.950054 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:11:12.950180 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.950183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.950835 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.950836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.951614 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:11:12.951616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.952173 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.952174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:11:12.952252 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.952257 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:11:12.952311 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:11:12.952361 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
I20260811 11:11:12.952461 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:12.949620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:45521 (local address 127.29.145.1:45305)
0811 11:11:12.949675 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:12.949678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:12.949834 (+ 156us) server_negotiation.cc:407] Connection header received
0811 11:11:12.949956 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:12.949957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:12.949979 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:12.950047 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:11:12.950266 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.950744 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.951719 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:12.952085 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:12.952172 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:12.952180 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:12.952335 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 11:11:12.952391 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:11:12.952523 10000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b020819a714eada4aff9248811b6ce" candidate_uuid: "4dfae5d1959c4313a7465a644a4afa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4ebeeabec04e3e8633c0a72572816b" is_pre_election: true
I20260811 11:11:12.952723 10000 raft_consensus.cc:2454] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dfae5d1959c4313a7465a644a4afa49 in term 0.
I20260811 11:11:12.952761 9692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b020819a714eada4aff9248811b6ce" candidate_uuid: "4dfae5d1959c4313a7465a644a4afa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919f4925daa148aaa812ab69a0170833" is_pre_election: true
I20260811 11:11:12.952898 9692 raft_consensus.cc:2454] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dfae5d1959c4313a7465a644a4afa49 in term 0.
I20260811 11:11:12.952972 9770 leader_election.cc:302] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c4ebeeabec04e3e8633c0a72572816b, 4dfae5d1959c4313a7465a644a4afa49; no voters:
I20260811 11:11:12.953125 10085 raft_consensus.cc:2781] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:11:12.953181 10085 raft_consensus.cc:482] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:11:12.953212 10085 raft_consensus.cc:3037] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:12.953994 10085 raft_consensus.cc:504] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.954152 10085 leader_election.cc:288] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [CANDIDATE]: Term 1 election: Requested vote from peers 1c4ebeeabec04e3e8633c0a72572816b (127.29.145.3:35751), 919f4925daa148aaa812ab69a0170833 (127.29.145.1:45305)
I20260811 11:11:12.954332 9692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b020819a714eada4aff9248811b6ce" candidate_uuid: "4dfae5d1959c4313a7465a644a4afa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919f4925daa148aaa812ab69a0170833"
I20260811 11:11:12.954335 10000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76b020819a714eada4aff9248811b6ce" candidate_uuid: "4dfae5d1959c4313a7465a644a4afa49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4ebeeabec04e3e8633c0a72572816b"
I20260811 11:11:12.954409 10000 raft_consensus.cc:3037] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:12.954427 9692 raft_consensus.cc:3037] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:11:12.955123 9692 raft_consensus.cc:2454] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dfae5d1959c4313a7465a644a4afa49 in term 1.
I20260811 11:11:12.955238 10000 raft_consensus.cc:2454] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dfae5d1959c4313a7465a644a4afa49 in term 1.
I20260811 11:11:12.955364 9771 leader_election.cc:302] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dfae5d1959c4313a7465a644a4afa49, 919f4925daa148aaa812ab69a0170833; no voters:
I20260811 11:11:12.955477 10085 raft_consensus.cc:2781] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:11:12.955693 10085 raft_consensus.cc:686] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 1 LEADER]: Becoming Leader. State: Replica: 4dfae5d1959c4313a7465a644a4afa49, State: Running, Role: LEADER
I20260811 11:11:12.955813 10085 consensus_queue.cc:237] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } }
I20260811 11:11:12.956733 9526 catalog_manager.cc:5495] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dfae5d1959c4313a7465a644a4afa49 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "4dfae5d1959c4313a7465a644a4afa49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dfae5d1959c4313a7465a644a4afa49" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37971 } health_report { overall_health: HEALTHY } } }
I20260811 11:11:13.015202 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:13.012402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.29.145.2:37971)
0811 11:11:13.012483 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 11:11:13.012485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:11:13.012492 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:11:13.012529 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:11:13.012530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:11:13.012550 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:11:13.012627 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:11:13.012832 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:13.013302 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:13.014291 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:11:13.014565 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:11:13.014645 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:11:13.014837 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:11:13.014952 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:11:13.015045 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:11:13.015069 (+ 24us) server_negotiation.cc:299] Negotiation successful
0811 11:11:13.015091 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:11:13.017335 9692 raft_consensus.cc:1261] T 76b020819a714eada4aff9248811b6ce P 919f4925daa148aaa812ab69a0170833 [term 1 FOLLOWER]: Refusing update from remote peer 4dfae5d1959c4313a7465a644a4afa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:11:13.017335 10000 raft_consensus.cc:1261] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 1 FOLLOWER]: Refusing update from remote peer 4dfae5d1959c4313a7465a644a4afa49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:11:13.017782 10087 consensus_queue.cc:1035] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c4ebeeabec04e3e8633c0a72572816b" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:11:13.017992 10085 consensus_queue.cc:1035] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "919f4925daa148aaa812ab69a0170833" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 45305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:11:13.020632 10090 mvcc.cc:205] Tried to move back new op lower bound from 7317285572675530752 to 7317285572427644928. Current Snapshot: MvccSnapshot[applied={T|T < 7317285572675530752}]
I20260811 11:11:13.021713 10091 mvcc.cc:205] Tried to move back new op lower bound from 7317285572675530752 to 7317285572427644928. Current Snapshot: MvccSnapshot[applied={T|T < 7317285572675530752 or (T in {7317285572675530752})}]
I20260811 11:11:13.023144 9526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:11:13.023252 9526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:11:13.024173 9526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edc43acf24a94d8aa0414625941c2b4b: Sending DeleteTablet for 3 replicas of tablet 76b020819a714eada4aff9248811b6ce
I20260811 11:11:13.024565 9666 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b020819a714eada4aff9248811b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:11:13 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:40054
I20260811 11:11:13.024653 9978 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b020819a714eada4aff9248811b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:11:13 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:53978
I20260811 11:11:13.024830 9825 tablet_service.cc:1514] Processing DeleteTablet for tablet 76b020819a714eada4aff9248811b6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:11:13 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:39236
I20260811 11:11:13.024870 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9595
I20260811 11:11:13.027451 10106 tablet_replica.cc:330] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: stopping tablet replica
I20260811 11:11:13.027571 10106 raft_consensus.cc:2229] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:11:13.027681 10106 raft_consensus.cc:2258] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:11:13.028208 10106 ts_tablet_manager.cc:1905] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:11:13.030498 10106 ts_tablet_manager.cc:1918] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:11:13.030607 10106 log.cc:1196] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/76b020819a714eada4aff9248811b6ce
I20260811 11:11:13.030910 10106 ts_tablet_manager.cc:1939] T 76b020819a714eada4aff9248811b6ce P 4dfae5d1959c4313a7465a644a4afa49: Deleting consensus metadata
I20260811 11:11:13.031409 9508 catalog_manager.cc:4841] TS 4dfae5d1959c4313a7465a644a4afa49 (127.29.145.2:37971): tablet 76b020819a714eada4aff9248811b6ce (table test-table [id=6e18916ea89c4dc68528075bad3409d9]) successfully deleted
W20260811 11:11:13.035477 9509 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:45305 (error 108)
I20260811 11:11:13.035665 10105 tablet_replica.cc:330] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: stopping tablet replica
I20260811 11:11:13.035724 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9751
I20260811 11:11:13.035809 10105 raft_consensus.cc:2229] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:11:13.035948 10105 raft_consensus.cc:2258] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:11:13.037806 10105 ts_tablet_manager.cc:1905] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:11:13.038753 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:11:13.038598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:45305 (local address 127.0.0.1:40068)
0811 11:11:13.038670 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 11:11:13.038719 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:45305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 11:11:13.038883 9509 catalog_manager.cc:4584] TS 919f4925daa148aaa812ab69a0170833 (127.29.145.1:45305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76b020819a714eada4aff9248811b6ce: Network error: Client connection negotiation failed: client connection to 127.29.145.1:45305: connect: Connection refused (error 111)
I20260811 11:11:13.040144 10105 ts_tablet_manager.cc:1918] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:11:13.040221 10105 log.cc:1196] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/76b020819a714eada4aff9248811b6ce
I20260811 11:11:13.040529 10105 ts_tablet_manager.cc:1939] T 76b020819a714eada4aff9248811b6ce P 1c4ebeeabec04e3e8633c0a72572816b: Deleting consensus metadata
I20260811 11:11:13.041378 9508 catalog_manager.cc:4841] TS 1c4ebeeabec04e3e8633c0a72572816b (127.29.145.3:35751): tablet 76b020819a714eada4aff9248811b6ce (table test-table [id=6e18916ea89c4dc68528075bad3409d9]) successfully deleted
I20260811 11:11:13.043680 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9904
I20260811 11:11:13.052187 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 9489
2026-08-11T11:11:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2411 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:11:13 dist-test-slave-69rg krb5kdc[10110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:11:13 dist-test-slave-69rg krb5kdc[10110](info): set up 2 sockets
Aug 11 11:11:13 dist-test-slave-69rg krb5kdc[10110](info): commencing operation
krb5kdc: starting...
W20260811 11:11:14.204257 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:11:14 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:11:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangeradmin_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangeradmin_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangerlookup_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangerlookup_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
I20260811 11:11:14.395108 30276 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_kobm8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/postgres -l logfile start
2026-08-11 11:11:17.414 UTC [10147] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 11:11:17.414 UTC [10147] LOG: listening on IPv4 address "127.29.145.20", port 39495
2026-08-11 11:11:17.419 UTC [10147] LOG: listening on Unix socket "/tmp/.s.PGSQL.39495"
2026-08-11 11:11:17.444 UTC [10150] LOG: database system was shut down at 2026-08-11 11:11:15 UTC
2026-08-11 11:11:17.448 UTC [10147] LOG: database system is ready to accept connections
I20260811 11:11:18.478758 30276 mini_postgres.cc:96] Postgres bound to 39495
2026-08-11 11:11:18.487 UTC [10159] FATAL: database "slave" does not exist
127.29.145.20:39495 - accepting connections
I20260811 11:11:18.487884 30276 mini_ranger.cc:161] Starting Ranger...
I20260811 11:11:18.504376 30276 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 11:11:19.119256 30276 mini_ranger.cc:87] Created ranger Postgres database
I20260811 11:11:19.119379 30276 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin
2026-08-11 11:11:19,487 [I] DB FLAVOR :POSTGRES
2026-08-11 11:11:19,488 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:11:19,488 [I] Checking connection..
2026-08-11 11:11:19,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 11:11:19,765 [I] Checking connection passed.
2026-08-11 11:11:19,765 [I] --------- Verifying version history table ---------
2026-08-11 11:11:19,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:11:20,044 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:11:20,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:11:20,314 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:11:20,314 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 11:11:20,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 11:11:20.583 UTC [10235] WARNING: there is no transaction in progress
2026-08-11 11:11:20,592 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 11:11:20,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:11:20,848 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 11:11:20,849 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:11:20,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 11:11:21,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 11:11:21,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-69rg.c.gcp-upstream.internal','N') ;"
2026-08-11 11:11:21,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:11:21,946 [I] Table x_portal_user does not exist in database ranger
2026-08-11 11:11:21,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:11:22,210 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 11:11:22,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:11:22,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:11:22,756 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 11:11:22,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 11:11:23.837 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:23.857 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:23.876 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:23.894 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:23.913 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:23.935 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.001 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.010 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.029 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.040 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.051 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.073 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.084 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.091 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.098 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24.106 UTC [10361] WARNING: there is no transaction in progress
2026-08-11 11:11:24,882 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 11:11:24,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 11:11:25,116 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 11:11:25,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:11:25,377 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 11:11:25,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:11:25,652 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 11:11:25,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:11:25,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:11:26,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-69rg.c.gcp-upstream.internal';"
2026-08-11 11:11:26,439 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 11:11:26,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 11:11:26,694 [I] DB_PATCHES have already been applied
I20260811 11:11:26.700925 30276 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_kobm8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 11:11:26.701007 30276 mini_ranger.cc:193] Using host: 127.29.145.20
I20260811 11:11:26.705466 30276 mini_ranger.cc:227] Ranger admin URL: http://127.29.145.20:57711
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 11:11:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 11:11:27.895534 30276 mini_ranger.cc:160] Time spent starting Ranger: real 9.408s user 0.000s sys 0.013s
Aug 11, 2026 11:11:49 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 11:11:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 11:11:49 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 11:11:49 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 11:11:50 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 11:11:50.856123 30276 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangerkms_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/rangerkms_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.145.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.145.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 11:11:50 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446710, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 11:11:50.968832 30276 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 11:11:50.980140 30276 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 11:11:52.080034 30276 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 11:11:52.080161 30276 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms
I20260811 11:11:52.080188 30276 mini_ranger_kms.cc:227] Using postgres at 127.29.145.20:39495
2026-08-11 11:11:52,405 [I] DB FLAVOR :POSTGRES
2026-08-11 11:11:52,405 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:11:52,405 [I] Checking connection
2026-08-11 11:11:52,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 11:11:52,704 [I] connection success
2026-08-11 11:11:52,704 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 11:11:52,704 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 11:11:52,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 11:11:52,970 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 11:11:52,970 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:11:52,970 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 11:11:52,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:39495/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 11:11:53,268 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 11:11:53.832054 30276 mini_ranger_kms.cc:310] Created kms service
I20260811 11:11:54.036589 30276 mini_ranger_kms.cc:326] Created kudu user
I20260811 11:11:54.084209 30276 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 11:11:54 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446710, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:11:54.524290 30276 mini_ranger_kms.cc:384] Added ranger policy
I20260811 11:11:54.524423 30276 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms:/tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 11:11:54.524449 30276 mini_ranger_kms.cc:240] Using host: 127.29.145.20
I20260811 11:11:54.528796 30276 mini_ranger_kms.cc:276] Ranger KMS PID: 10568
I20260811 11:11:54.528927 30276 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.145.20:45263
Aug 11, 2026 11:11:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:11:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:11:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 11:11:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 11:11:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 11:11:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 11:11:56 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786446716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:11:56.992018 30276 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.023s user 0.000s sys 0.300s
I20260811 11:11:56.992205 30276 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 11:11:56.992311 30276 mini_ranger_kms.cc:401] 127.29.145.20:45263/kms/v1/keys
Aug 11 11:11:57 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
Aug 11 11:11:57 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
Aug 11 11:11:59 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:11:59.957562 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39323
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:39323
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:45263/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:00.060529 10628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:00.060863 10628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:00.060901 10628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:00.063102 10628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:12:00.063169 10628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:00.063195 10628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:00.063213 10628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:12:00.063247 10628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:12:00.065778 10628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.145.20:45263/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:39323
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:00.066097 10628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:00.066358 10628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:00.072279 10637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:00.072383 10635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:00.072324 10628 server_base.cc:1034] running on GCE node
W20260811 11:12:00.072286 10634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:00.072887 10628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:00.073275 10628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:00.074465 10628 hybrid_clock.cc:648] HybridClock initialized: now 1786446720074450 us; error 31 us; skew 500 ppm
Aug 11 11:12:00 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446720, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:00.082518 10628 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:12:00.083058 10628 webserver.cc:460] Webserver started at http://127.29.145.62:39861/ using document root <none> and password file <none>
I20260811 11:12:00.083307 10628 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:00.083487 10628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:00.083673 10628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:00 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446720, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:00.145926 10628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "eadc548d845b40c89c3d2f8c6193c18a"
format_stamp: "Formatted at 2026-08-11 11:12:00 on dist-test-slave-69rg"
server_key: "97f6a0b824a857dcc577238c769f7db4"
server_key_iv: "b267bc2d78fc25e604e2abc61bab2718"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:00.146405 10628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "eadc548d845b40c89c3d2f8c6193c18a"
format_stamp: "Formatted at 2026-08-11 11:12:00 on dist-test-slave-69rg"
server_key: "97f6a0b824a857dcc577238c769f7db4"
server_key_iv: "b267bc2d78fc25e604e2abc61bab2718"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:00.148556 10628 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:12:00.165361 10644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:00.165694 10628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:12:00.165807 10628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "eadc548d845b40c89c3d2f8c6193c18a"
format_stamp: "Formatted at 2026-08-11 11:12:00 on dist-test-slave-69rg"
server_key: "97f6a0b824a857dcc577238c769f7db4"
server_key_iv: "b267bc2d78fc25e604e2abc61bab2718"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:00.165982 10628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:00.172871 10628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:00.175069 10628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:00.175228 10628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:00.180397 10628 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:39323
I20260811 11:12:00.180421 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:39323 every 8 connection(s)
I20260811 11:12:00.180830 10628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:12:00.181646 10706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:00.184867 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: Bootstrap starting.
I20260811 11:12:00.185598 10706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:00.185921 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 10628
I20260811 11:12:00.186026 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:12:00.186208 30276 external_mini_cluster.cc:1428] Setting key bddc8a920e827df6ef5d09a65cb5579e
I20260811 11:12:00.186676 10706 log.cc:824] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:00.187737 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: No bootstrap required, opened a new log
I20260811 11:12:00.190374 10706 raft_consensus.cc:348] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } }
I20260811 11:12:00.190557 10706 raft_consensus.cc:374] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:00.190589 10706 raft_consensus.cc:729] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eadc548d845b40c89c3d2f8c6193c18a, State: Initialized, Role: FOLLOWER
I20260811 11:12:00.190743 10706 consensus_queue.cc:260] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } }
I20260811 11:12:00.190834 10706 raft_consensus.cc:388] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:12:00.190864 10706 raft_consensus.cc:482] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:12:00.190932 10706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:00.191752 10706 raft_consensus.cc:504] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } }
I20260811 11:12:00.191879 10706 leader_election.cc:302] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eadc548d845b40c89c3d2f8c6193c18a; no voters:
I20260811 11:12:00.192056 10706 leader_election.cc:288] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:12:00.192203 10712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:12:00.192327 10706 sys_catalog.cc:564] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:12:00.192425 10712 raft_consensus.cc:686] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [term 1 LEADER]: Becoming Leader. State: Replica: eadc548d845b40c89c3d2f8c6193c18a, State: Running, Role: LEADER
I20260811 11:12:00.192600 10712 consensus_queue.cc:237] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } }
I20260811 11:12:00.193114 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eadc548d845b40c89c3d2f8c6193c18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } } }
I20260811 11:12:00.193271 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:00.193497 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eadc548d845b40c89c3d2f8c6193c18a. Latest consensus state: current_term: 1 leader_uuid: "eadc548d845b40c89c3d2f8c6193c18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eadc548d845b40c89c3d2f8c6193c18a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39323 } } }
I20260811 11:12:00.193653 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:00.193856 10723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 11:12:00 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:00.194360 10706 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:12:00.194504 10723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:12:00.202703 10723 catalog_manager.cc:1347] Generated new cluster ID: 794e8dc498924bb686777a843bcb4c65
I20260811 11:12:00.202834 10723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:12:00.204609 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:00.186783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.29.145.62:39323)
0811 11:12:00.187077 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:00.187088 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:00.187127 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:12:00.187706 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:00.187714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:00.187823 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:00.187986 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:00.188733 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:00.189720 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:00.190785 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:00.191406 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:00.191545 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:00.194926 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:00.194931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:00.194931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:00.194947 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:00.201649 (+ 6702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:00.202222 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:00.202230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:00.202231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:00.202423 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:00.202692 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:00.202696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:00.202697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:00.203232 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:00.203895 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:00.203975 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 11:12:00.204285 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":154,"threads_started":1}
I20260811 11:12:00.223560 10723 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:12:00.224376 10723 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:12:00.234417 10723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: Generated new TSK 0
I20260811 11:12:00.234746 10723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:12:01.034809 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:45263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:01.161391 10757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:01.161644 10757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:01.161675 10757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:01.163224 10757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:01.163297 10757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:01.163344 10757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:12:01.165555 10757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.145.20:45263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:01.165875 10757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:01.166213 10757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:01.173094 10762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:01.173126 10757 server_base.cc:1034] running on GCE node
W20260811 11:12:01.173269 10765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:01.173094 10763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:01.173669 10757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:01.174005 10757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:01.175206 10757 hybrid_clock.cc:648] HybridClock initialized: now 1786446721175191 us; error 34 us; skew 500 ppm
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:01.178189 10757 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:12:01.178771 10757 webserver.cc:460] Webserver started at http://127.29.145.1:38191/ using document root <none> and password file <none>
I20260811 11:12:01.179031 10757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:01.179101 10757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:01.179258 10757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:01.199913 10757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "c5a13b02d76b4168bdc16f810e5b30a4"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "888bfd4dc3ec29818492893cb8af4217"
server_key_iv: "9981d6fc727531df0f2514805068001a"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.200361 10757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "c5a13b02d76b4168bdc16f810e5b30a4"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "888bfd4dc3ec29818492893cb8af4217"
server_key_iv: "9981d6fc727531df0f2514805068001a"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.202132 10757 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:12:01.218276 10775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.218559 10757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:01.218670 10757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "c5a13b02d76b4168bdc16f810e5b30a4"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "888bfd4dc3ec29818492893cb8af4217"
server_key_iv: "9981d6fc727531df0f2514805068001a"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.218783 10757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:01.240448 10757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:01.243280 10757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:01.243510 10757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:01.243821 10757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:01.244289 10757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:01.244393 10757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.244516 10757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:01.244596 10757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.253304 10757 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:35179
I20260811 11:12:01.253393 10908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:35179 every 8 connection(s)
I20260811 11:12:01.253931 10757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:12:01.254788 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 10757
I20260811 11:12:01.254930 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:12:01.255174 30276 external_mini_cluster.cc:1428] Setting key a2a1d767e9c603abaeb8a3169285683d
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:01.268118 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.255363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:53839 (local address 127.29.145.62:39323)
0811 11:12:01.258250 (+ 2887us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.258253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.258264 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:12:01.258294 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.258294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.258309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.258396 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:01.259295 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.260105 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.261704 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.262055 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.262132 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.264788 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:01.264790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:01.264790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:01.264799 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:01.266551 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.267034 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.267036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.267037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.267110 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.267379 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.267381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.267381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.267516 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:01.267590 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.267912 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.268003 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2858,"thread_start_us":63,"threads_started":1}
I20260811 11:12:01.268321 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.255005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39323 (local address 127.29.145.1:53839)
0811 11:12:01.255206 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.255253 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.255935 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.258562 (+ 2627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.258567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.258846 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:01.259169 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.259177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.260234 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.260235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.261583 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.261587 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.262210 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.262212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.262363 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.262990 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:01.263002 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:01.264663 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:01.266675 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.266684 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.266687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.266938 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.267186 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.267187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.267188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.267309 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.267592 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:01.267608 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:01.267764 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.267890 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.268058 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260811 11:12:01.268896 10909 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39323
I20260811 11:12:01.269022 10909 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:01.269291 10909 heartbeater.cc:511] Master 127.29.145.62:39323 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
I20260811 11:12:01.269974 10663 ts_manager.cc:194] Registered new tserver with Master: c5a13b02d76b4168bdc16f810e5b30a4 (127.29.145.1:35179)
I20260811 11:12:01.270789 10663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:53839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:12:01.293970 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:45263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 11:12:01.422451 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:01.422780 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:01.422844 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:01.424675 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:01.424748 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:01.424790 10916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:12:01.427273 10916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.145.20:45263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:01.427693 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:01.428012 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:01.434520 10923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:01.434922 10924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:01.438755 10926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:01.439802 10916 server_base.cc:1034] running on GCE node
I20260811 11:12:01.440141 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:01.440459 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:01.443746 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786446721443717 us; error 40 us; skew 500 ppm
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:01.447472 10916 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:12:01.448052 10916 webserver.cc:460] Webserver started at http://127.29.145.2:39343/ using document root <none> and password file <none>
I20260811 11:12:01.448366 10916 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:01.448423 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:01.448560 10916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:01.465512 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "fd46aa03d2194168be49dd8cd8d79a24"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "242b1675ecf52be98352025229680f3d"
server_key_iv: "9c870293ea02542839041c553685d50f"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.466069 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "fd46aa03d2194168be49dd8cd8d79a24"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "242b1675ecf52be98352025229680f3d"
server_key_iv: "9c870293ea02542839041c553685d50f"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.468000 10916 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 11:12:01.481508 10935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.481808 10916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:12:01.481948 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "fd46aa03d2194168be49dd8cd8d79a24"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "242b1675ecf52be98352025229680f3d"
server_key_iv: "9c870293ea02542839041c553685d50f"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.482041 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:01.510713 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:01.513401 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:01.513622 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:01.513978 10916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:01.514643 10916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:01.514741 10916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.514817 10916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:01.514883 10916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.521473 10916 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:37265
I20260811 11:12:01.521996 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:12:01.525013 11064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:37265 every 8 connection(s)
I20260811 11:12:01.530366 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 10916
I20260811 11:12:01.530476 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:12:01.530678 30276 external_mini_cluster.cc:1428] Setting key 0e013c5fc6df01c3a978287803422517
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
I20260811 11:12:01.546054 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.525749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:38829 (local address 127.29.145.62:39323)
0811 11:12:01.525796 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.525798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.526085 (+ 287us) server_negotiation.cc:407] Connection header received
0811 11:12:01.526739 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.526741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.526759 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.526833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:01.527538 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.528321 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.529845 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.530182 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.530260 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.533696 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:01.533698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:01.533698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:01.533706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:01.535419 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.544833 (+ 9414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.544835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.544836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.544912 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.545286 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.545288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.545288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.545432 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:01.545513 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.545845 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.545957 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:12:01.546399 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.525721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39323 (local address 127.29.145.2:38829)
0811 11:12:01.525923 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.525975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.526608 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.526900 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.526905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.527152 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:01.527421 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.527427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.528444 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.528445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.529713 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.529716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.530336 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.530337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.530485 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.531090 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:01.531100 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:01.533555 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:01.544436 (+ 10881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.544447 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.544450 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.544709 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.545002 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.545003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.545004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.545221 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.545513 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:01.545521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:01.545681 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.545800 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.545990 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260811 11:12:01.546882 11065 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
I20260811 11:12:01.546985 11065 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:01.547207 11065 heartbeater.cc:511] Master 127.29.145.62:39323 requested a full tablet report, sending...
I20260811 11:12:01.547674 10663 ts_manager.cc:194] Registered new tserver with Master: fd46aa03d2194168be49dd8cd8d79a24 (127.29.145.2:37265)
I20260811 11:12:01.548166 10663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:38829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:12:01.569556 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:45263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:01.699268 11071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:01.699570 11071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:01.699617 11071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:01.700896 11071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:01.700944 11071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:01.700973 11071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:12:01.702729 11071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.145.20:45263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:01.702977 11071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:01.703256 11071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:01.708644 11077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:01.708611 11080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:01.708618 11078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:01.710156 11071 server_base.cc:1034] running on GCE node
I20260811 11:12:01.710443 11071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:01.710745 11071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:01.711918 11071 hybrid_clock.cc:648] HybridClock initialized: now 1786446721711896 us; error 45 us; skew 500 ppm
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:01.714859 11071 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:12:01.715402 11071 webserver.cc:460] Webserver started at http://127.29.145.3:43353/ using document root <none> and password file <none>
I20260811 11:12:01.715679 11071 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:01.715726 11071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:01.715904 11071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:01.729895 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "82a1ab7ba82c4f3fa3c51a3949c55844"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "f29bbdeb8b044048f639fc2b74f99fc2"
server_key_iv: "5d81e36b1709f58ab4d25e25dcc22f65"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.730386 11071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "82a1ab7ba82c4f3fa3c51a3949c55844"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "f29bbdeb8b044048f639fc2b74f99fc2"
server_key_iv: "5d81e36b1709f58ab4d25e25dcc22f65"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.742409 11071 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.000s sys 0.002s
I20260811 11:12:01.757328 11089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.757577 11071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:12:01.757685 11071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "82a1ab7ba82c4f3fa3c51a3949c55844"
format_stamp: "Formatted at 2026-08-11 11:12:01 on dist-test-slave-69rg"
server_key: "f29bbdeb8b044048f639fc2b74f99fc2"
server_key_iv: "5d81e36b1709f58ab4d25e25dcc22f65"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:01.757797 11071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:01.772857 11071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:01.775561 11071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:01.775791 11071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:01.776100 11071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:01.776558 11071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:01.776644 11071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.776710 11071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:01.776757 11071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:01.784554 11071 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:36571
I20260811 11:12:01.784687 11225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:36571 every 8 connection(s)
I20260811 11:12:01.785125 11071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:12:01.788132 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 11071
I20260811 11:12:01.788250 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:12:01.788450 30276 external_mini_cluster.cc:1428] Setting key d8b197c1a12e6a62dc13d6015ed3b5e8
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:01.798048 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.785999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:51491 (local address 127.29.145.62:39323)
0811 11:12:01.786115 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.786117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.786453 (+ 336us) server_negotiation.cc:407] Connection header received
0811 11:12:01.787138 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.787139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.787153 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.787256 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:01.788813 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.789579 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.791172 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.791464 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.791513 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.794413 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:01.794415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:01.794415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:01.794423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:01.796256 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.796820 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.796821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.796822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.796888 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.797199 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.797200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.797200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.797349 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:01.797397 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.797857 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.797950 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:12:01.798671 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.785998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39323 (local address 127.29.145.3:51491)
0811 11:12:01.786295 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.786339 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.787016 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.787337 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.787341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.787606 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:01.787914 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.787920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.789720 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.789721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.791056 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.791059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.791629 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.791630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.791779 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.792430 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:01.792442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:01.794296 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:01.796365 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.796375 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.796378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.796718 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.796965 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:01.796966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:01.796966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:01.797114 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:01.797450 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:01.797457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:01.797655 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.798193 (+ 538us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.798379 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":100,"threads_started":1}
I20260811 11:12:01.799207 11226 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39323
I20260811 11:12:01.799319 11226 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:01.799532 11226 heartbeater.cc:511] Master 127.29.145.62:39323 requested a full tablet report, sending...
I20260811 11:12:01.799912 10663 ts_manager.cc:194] Registered new tserver with Master: 82a1ab7ba82c4f3fa3c51a3949c55844 (127.29.145.3:36571)
I20260811 11:12:01.800415 10663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:51491
I20260811 11:12:01.800922 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:12:01.811144 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.802927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.29.145.62:39323)
0811 11:12:01.802973 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.802975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.803007 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:12:01.803028 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.803029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.803039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.803099 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:01.803862 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.804604 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.805701 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.806091 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.806142 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.807928 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:01.807930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:01.807930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:01.807940 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:01.809744 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.810210 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.810211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.810212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.810278 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:01.810511 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:01.810512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:01.810512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:01.810659 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:01.810759 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.810967 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.811049 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:12:01.813488 10663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:12:01.816219 10663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:12:01.830789 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.820491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:37265 (local address 127.0.0.1:46120)
0811 11:12:01.825911 (+ 5420us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.825923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.825989 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.826272 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.826276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.826279 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:01.826363 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.826365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.827704 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.827705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.829266 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.829269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.830424 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.830425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.830518 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.830535 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.830616 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.830692 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5400,"thread_start_us":82,"threads_started":1}
I20260811 11:12:01.831271 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.821015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.29.145.2:37265)
0811 11:12:01.821197 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.821204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.826065 (+ 4861us) server_negotiation.cc:407] Connection header received
0811 11:12:01.826127 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.826136 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.826153 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.826245 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:01.826449 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.827582 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.829362 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.830216 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.830297 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.830383 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.831133 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.831183 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260811 11:12:01.832329 10991 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 (DEFAULT_TABLE table=test-table [id=6e2e3ab361bb47f5bc1b8a60bef92896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:01.832780 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe04d689aa1492aa67b0a1d5c635258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:01.834343 11242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.820049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35179 (local address 127.0.0.1:45980)
0811 11:12:01.820463 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.820506 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.820584 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.830150 (+ 9566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.830152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.830154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:01.830204 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.830206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.831904 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.831905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.832995 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.832998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.834129 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.834131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.834196 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.834201 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.834231 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.834271 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":90,"threads_started":1}
I20260811 11:12:01.834687 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.820163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.29.145.1:35179)
0811 11:12:01.829959 (+ 9796us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.829964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.829977 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:12:01.830015 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.830018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.830035 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.830276 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:01.830340 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.831347 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.833109 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.834018 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.834457 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.834528 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.834555 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.834597 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9751,"thread_start_us":67,"threads_started":1}
I20260811 11:12:01.837160 11246 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Bootstrap starting.
I20260811 11:12:01.838220 11246 tablet_bootstrap.cc:654] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:01.838289 10815 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 (DEFAULT_TABLE table=test-table [id=6e2e3ab361bb47f5bc1b8a60bef92896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:01.838506 11246 log.cc:824] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:01.838673 10815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe04d689aa1492aa67b0a1d5c635258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:01.842502 11246 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: No bootstrap required, opened a new log
I20260811 11:12:01.842839 11246 ts_tablet_manager.cc:1397] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260811 11:12:01.844856 11246 raft_consensus.cc:348] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.845171 11246 raft_consensus.cc:374] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:01.845252 11246 raft_consensus.cc:729] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd46aa03d2194168be49dd8cd8d79a24, State: Initialized, Role: FOLLOWER
I20260811 11:12:01.845479 11246 consensus_queue.cc:260] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.845903 11246 ts_tablet_manager.cc:1428] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 11:12:01.846078 11065 heartbeater.cc:503] Master 127.29.145.62:39323 was elected leader, sending a full tablet report...
I20260811 11:12:01.847592 11249 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Bootstrap starting.
I20260811 11:12:01.847685 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.820921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.29.145.3:36571)
0811 11:12:01.831554 (+ 10633us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:01.831559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:01.831573 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:12:01.831660 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:01.831664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:01.831684 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:01.831778 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:01.843664 (+ 11886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.844840 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.846347 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.847120 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.847175 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.847257 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:01.847510 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 11:12:01.847557 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10577,"thread_start_us":100,"threads_started":1}
I20260811 11:12:01.848438 11249 tablet_bootstrap.cc:654] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:01.848471 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:01.820288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36571 (local address 127.0.0.1:57704)
0811 11:12:01.821450 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:01.821462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:01.821520 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:01.843449 (+ 21929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:01.843451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:01.843455 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:01.843519 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.843522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.845000 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.845001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.846165 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:01.846170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:01.847334 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:01.847335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:01.847407 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:01.847410 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:12:01.848325 (+ 915us) client_negotiation.cc:240] Negotiation successful
0811 11:12:01.848382 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":87,"threads_started":1}
I20260811 11:12:01.848809 11249 log.cc:824] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:01.849406 11140 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 (DEFAULT_TABLE table=test-table [id=6e2e3ab361bb47f5bc1b8a60bef92896]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:01.849649 11249 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: No bootstrap required, opened a new log
I20260811 11:12:01.849709 11249 ts_tablet_manager.cc:1397] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:01.849792 11140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe04d689aa1492aa67b0a1d5c635258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:01.852308 11249 raft_consensus.cc:348] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.852469 11249 raft_consensus.cc:374] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:01.852497 11249 raft_consensus.cc:729] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a13b02d76b4168bdc16f810e5b30a4, State: Initialized, Role: FOLLOWER
I20260811 11:12:01.852638 11249 consensus_queue.cc:260] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.852922 11249 ts_tablet_manager.cc:1428] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:12:01.853044 10909 heartbeater.cc:503] Master 127.29.145.62:39323 was elected leader, sending a full tablet report...
I20260811 11:12:01.855540 11254 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Bootstrap starting.
I20260811 11:12:01.856922 11254 tablet_bootstrap.cc:654] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:01.857429 11254 log.cc:824] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:01.858543 11254 tablet_bootstrap.cc:492] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: No bootstrap required, opened a new log
I20260811 11:12:01.858636 11254 ts_tablet_manager.cc:1397] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:12:01.860883 11254 raft_consensus.cc:348] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.861112 11254 raft_consensus.cc:374] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:01.861157 11254 raft_consensus.cc:729] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82a1ab7ba82c4f3fa3c51a3949c55844, State: Initialized, Role: FOLLOWER
I20260811 11:12:01.861307 11254 consensus_queue.cc:260] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:01.861667 11254 ts_tablet_manager.cc:1428] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:12:01.861735 11226 heartbeater.cc:503] Master 127.29.145.62:39323 was elected leader, sending a full tablet report...
Aug 11 11:12:01 dist-test-slave-69rg krb5kdc[10110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446721, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
W20260811 11:12:02.006450 10910 tablet.cc:2369] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:12:02.025929 11066 tablet.cc:2369] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:12:02.036013 11227 tablet.cc:2369] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:12:02.089527 11251 raft_consensus.cc:482] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:12:02.089699 11251 raft_consensus.cc:504] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:02.090122 11251 leader_election.cc:288] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5a13b02d76b4168bdc16f810e5b30a4 (127.29.145.1:35179), 82a1ab7ba82c4f3fa3c51a3949c55844 (127.29.145.3:36571)
I20260811 11:12:02.094422 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.090715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:38319 (local address 127.29.145.3:36571)
0811 11:12:02.090758 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:02.090759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:02.090868 (+ 109us) server_negotiation.cc:407] Connection header received
0811 11:12:02.090975 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:02.090976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:02.090985 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:02.091015 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:02.091276 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.091991 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.093441 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.093999 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.094054 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:02.094064 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:02.094294 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 11:12:02.094335 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:12:02.094620 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.090456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:53891 (local address 127.29.145.1:35179)
0811 11:12:02.090505 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:02.090507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:02.090663 (+ 156us) server_negotiation.cc:407] Connection header received
0811 11:12:02.090771 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:02.090772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:02.090784 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:02.090813 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:02.091022 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.091769 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.093540 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.094223 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.094274 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:02.094281 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:02.094501 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 11:12:02.094541 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:12:02.094874 11259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.090561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36571 (local address 127.29.145.2:38319)
0811 11:12:02.090802 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:02.090812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:02.090895 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:02.091107 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:02.091107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:02.091109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:02.091185 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:02.091187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.092141 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.092142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:02.093326 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:02.093329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.094140 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.094141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:02.094212 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:02.094215 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:12:02.094740 (+ 525us) client_negotiation.cc:240] Negotiation successful
0811 11:12:02.094788 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":41,"threads_started":1}
I20260811 11:12:02.095012 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.090352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35179 (local address 127.29.145.2:53891)
0811 11:12:02.090572 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:02.090584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:02.090693 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:02.090879 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:02.090880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:02.090882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:02.090951 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:02.090952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.091875 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.091876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:02.093302 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:02.093305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.094342 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.094342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:02.094425 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:02.094428 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:12:02.094886 (+ 458us) client_negotiation.cc:240] Negotiation successful
0811 11:12:02.094932 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":48,"thread_start_us":89,"threads_started":1}
I20260811 11:12:02.095626 11168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe04d689aa1492aa67b0a1d5c635258" candidate_uuid: "fd46aa03d2194168be49dd8cd8d79a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" is_pre_election: true
I20260811 11:12:02.095813 11168 raft_consensus.cc:2454] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd46aa03d2194168be49dd8cd8d79a24 in term 0.
I20260811 11:12:02.096095 10938 leader_election.cc:302] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82a1ab7ba82c4f3fa3c51a3949c55844, fd46aa03d2194168be49dd8cd8d79a24; no voters:
I20260811 11:12:02.096324 11251 raft_consensus.cc:2781] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:12:02.096391 11251 raft_consensus.cc:482] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:12:02.096421 11251 raft_consensus.cc:3037] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:02.097385 11251 raft_consensus.cc:504] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:02.097527 11251 leader_election.cc:288] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [CANDIDATE]: Term 1 election: Requested vote from peers c5a13b02d76b4168bdc16f810e5b30a4 (127.29.145.1:35179), 82a1ab7ba82c4f3fa3c51a3949c55844 (127.29.145.3:36571)
I20260811 11:12:02.097705 10855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe04d689aa1492aa67b0a1d5c635258" candidate_uuid: "fd46aa03d2194168be49dd8cd8d79a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" is_pre_election: true
I20260811 11:12:02.097882 10855 raft_consensus.cc:2454] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd46aa03d2194168be49dd8cd8d79a24 in term 0.
I20260811 11:12:02.097985 11168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe04d689aa1492aa67b0a1d5c635258" candidate_uuid: "fd46aa03d2194168be49dd8cd8d79a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844"
I20260811 11:12:02.098026 10853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe04d689aa1492aa67b0a1d5c635258" candidate_uuid: "fd46aa03d2194168be49dd8cd8d79a24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a13b02d76b4168bdc16f810e5b30a4"
I20260811 11:12:02.098115 10853 raft_consensus.cc:3037] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:02.098093 11168 raft_consensus.cc:3037] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:02.099023 11168 raft_consensus.cc:2454] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd46aa03d2194168be49dd8cd8d79a24 in term 1.
I20260811 11:12:02.099028 10853 raft_consensus.cc:2454] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd46aa03d2194168be49dd8cd8d79a24 in term 1.
I20260811 11:12:02.099308 10938 leader_election.cc:302] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82a1ab7ba82c4f3fa3c51a3949c55844, fd46aa03d2194168be49dd8cd8d79a24; no voters:
I20260811 11:12:02.099493 11251 raft_consensus.cc:2781] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:12:02.099690 11251 raft_consensus.cc:686] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 1 LEADER]: Becoming Leader. State: Replica: fd46aa03d2194168be49dd8cd8d79a24, State: Running, Role: LEADER
I20260811 11:12:02.099799 11251 consensus_queue.cc:237] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } }
I20260811 11:12:02.100718 10663 catalog_manager.cc:5495] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd46aa03d2194168be49dd8cd8d79a24 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "fd46aa03d2194168be49dd8cd8d79a24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd46aa03d2194168be49dd8cd8d79a24" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 } health_report { overall_health: UNKNOWN } } }
I20260811 11:12:02.121474 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.118117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.29.145.2:37265)
0811 11:12:02.118167 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:02.118169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:02.118178 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:12:02.118224 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:02.118225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:02.118250 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:02.118335 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:12:02.118501 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.119254 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.120398 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:02.120734 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:02.120810 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:02.121017 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:12:02.121171 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:12:02.121264 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:02.121325 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:12:02.121356 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:12:02.123818 11168 raft_consensus.cc:1261] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 1 FOLLOWER]: Refusing update from remote peer fd46aa03d2194168be49dd8cd8d79a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:12:02.123832 10853 raft_consensus.cc:1261] T 9fe04d689aa1492aa67b0a1d5c635258 P c5a13b02d76b4168bdc16f810e5b30a4 [term 1 FOLLOWER]: Refusing update from remote peer fd46aa03d2194168be49dd8cd8d79a24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:12:02.124346 11251 consensus_queue.cc:1035] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5a13b02d76b4168bdc16f810e5b30a4" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:12:02.124553 11261 consensus_queue.cc:1035] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82a1ab7ba82c4f3fa3c51a3949c55844" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:12:02.136657 11266 mvcc.cc:205] Tried to move back new op lower bound from 7317285773815959552 to 7317285773721296896. Current Snapshot: MvccSnapshot[applied={T|T < 7317285773815959552}]
I20260811 11:12:02.137164 11264 mvcc.cc:205] Tried to move back new op lower bound from 7317285773815959552 to 7317285773721296896. Current Snapshot: MvccSnapshot[applied={T|T < 7317285773815959552}]
I20260811 11:12:02.137336 11267 mvcc.cc:205] Tried to move back new op lower bound from 7317285773815959552 to 7317285773721296896. Current Snapshot: MvccSnapshot[applied={T|T < 7317285773815959552}]
I20260811 11:12:02.140750 10663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:12:02.140877 10663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:12:02.141937 10663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eadc548d845b40c89c3d2f8c6193c18a: Sending DeleteTablet for 3 replicas of tablet 9fe04d689aa1492aa67b0a1d5c635258
I20260811 11:12:02.142649 10815 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:02 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:45980
I20260811 11:12:02.142975 11140 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:02 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:57704
I20260811 11:12:02.143210 10991 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe04d689aa1492aa67b0a1d5c635258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:02 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:46120
I20260811 11:12:02.144021 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 10757
I20260811 11:12:02.149984 11277 tablet_replica.cc:330] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: stopping tablet replica
I20260811 11:12:02.150177 11277 raft_consensus.cc:2229] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:12:02.150324 11277 raft_consensus.cc:2258] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:12:02.150991 11277 ts_tablet_manager.cc:1905] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:12:02.155242 11277 ts_tablet_manager.cc:1918] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:12:02.155452 11277 log.cc:1196] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/9fe04d689aa1492aa67b0a1d5c635258
I20260811 11:12:02.156373 11277 ts_tablet_manager.cc:1939] T 9fe04d689aa1492aa67b0a1d5c635258 P 82a1ab7ba82c4f3fa3c51a3949c55844: Deleting consensus metadata
I20260811 11:12:02.157255 10647 catalog_manager.cc:4841] TS 82a1ab7ba82c4f3fa3c51a3949c55844 (127.29.145.3:36571): tablet 9fe04d689aa1492aa67b0a1d5c635258 (table test-table [id=6e2e3ab361bb47f5bc1b8a60bef92896]) successfully deleted
I20260811 11:12:02.161791 11278 tablet_replica.cc:330] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: stopping tablet replica
I20260811 11:12:02.162086 11278 raft_consensus.cc:2229] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:12:02.162385 11278 raft_consensus.cc:2258] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:12:02.163080 11278 ts_tablet_manager.cc:1905] T 9fe04d689aa1492aa67b0a1d5c635258 P fd46aa03d2194168be49dd8cd8d79a24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:12:02.164408 10647 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:35179 (error 108)
I20260811 11:12:02.165453 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 10916
I20260811 11:12:02.165462 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.165299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35179 (local address 127.0.0.1:45992)
0811 11:12:02.165365 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:02.165423 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 11:12:02.165602 10647 catalog_manager.cc:4584] TS c5a13b02d76b4168bdc16f810e5b30a4 (127.29.145.1:35179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fe04d689aa1492aa67b0a1d5c635258: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35179: connect: Connection refused (error 111)
I20260811 11:12:02.174938 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 11071
I20260811 11:12:02.174994 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:02.174848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:37265 (local address 127.0.0.1:46148)
0811 11:12:02.174923 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:02.174949 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.2:37265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 11:12:02.186309 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 10628
I20260811 11:12:02.195791 30276 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 11:12:02.541044 30276 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 11:12:02.541181 30276 mini_ranger.cc:66] Stopping Ranger...
I20260811 11:12:02.896703 30276 mini_ranger.cc:68] Stopped Ranger
2026-08-11 11:12:02.897 UTC [10147] LOG: received smart shutdown request
2026-08-11 11:12:02.902 UTC [10147] LOG: background worker "logical replication launcher" (PID 10156) exited with exit code 1
2026-08-11 11:12:02.902 UTC [10151] LOG: shutting down
2026-08-11 11:12:03.133 UTC [10147] LOG: database system is shut down
2026-08-11T11:12:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50212 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:12:03 dist-test-slave-69rg krb5kdc[11291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:12:03 dist-test-slave-69rg krb5kdc[11291](info): set up 2 sockets
Aug 11 11:12:03 dist-test-slave-69rg krb5kdc[11291](info): commencing operation
krb5kdc: starting...
W20260811 11:12:04.469141 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:12:04 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:12:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangeradmin_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangeradmin_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangerlookup_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangerlookup_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
I20260811 11:12:04.710457 30276 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_kobm8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/postgres -l logfile start
2026-08-11 11:12:09.253 UTC [11332] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 11:12:09.254 UTC [11332] LOG: listening on IPv4 address "127.29.145.20", port 37809
2026-08-11 11:12:09.265 UTC [11332] LOG: listening on Unix socket "/tmp/.s.PGSQL.37809"
2026-08-11 11:12:09.328 UTC [11336] LOG: database system was shut down at 2026-08-11 11:12:06 UTC
2026-08-11 11:12:09.337 UTC [11332] LOG: database system is ready to accept connections
I20260811 11:12:10.691015 30276 mini_postgres.cc:96] Postgres bound to 37809
2026-08-11 11:12:10.700 UTC [11348] FATAL: database "slave" does not exist
127.29.145.20:37809 - accepting connections
I20260811 11:12:10.701362 30276 mini_ranger.cc:161] Starting Ranger...
I20260811 11:12:10.726042 30276 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 11:12:11.854748 30276 mini_ranger.cc:87] Created ranger Postgres database
I20260811 11:12:11.854898 30276 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin
2026-08-11 11:12:12,231 [I] DB FLAVOR :POSTGRES
2026-08-11 11:12:12,233 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:12:12,233 [I] Checking connection..
2026-08-11 11:12:12,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 11:12:12,566 [I] Checking connection passed.
2026-08-11 11:12:12,566 [I] --------- Verifying version history table ---------
2026-08-11 11:12:12,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:12:12,926 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:12:12,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:12:13,316 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:12:13,316 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 11:12:13,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 11:12:13.697 UTC [11422] WARNING: there is no transaction in progress
2026-08-11 11:12:13,711 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 11:12:13,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:12:14,087 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 11:12:14,088 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:12:14,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 11:12:14,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 11:12:14,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-69rg.c.gcp-upstream.internal','N') ;"
2026-08-11 11:12:15,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:12:15,381 [I] Table x_portal_user does not exist in database ranger
2026-08-11 11:12:15,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:12:15,707 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 11:12:15,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:12:16,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:12:16,308 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 11:12:16,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 11:12:17.480 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.497 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.515 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.533 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.554 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.576 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.638 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.646 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.662 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.674 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.686 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.701 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.712 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.720 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.728 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:17.737 UTC [11546] WARNING: there is no transaction in progress
2026-08-11 11:12:18,556 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 11:12:18,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 11:12:18,821 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 11:12:18,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:12:19,085 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 11:12:19,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:12:19,350 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 11:12:19,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:12:19,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:12:19,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-69rg.c.gcp-upstream.internal';"
2026-08-11 11:12:20,134 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 11:12:20,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 11:12:20,416 [I] DB_PATCHES have already been applied
I20260811 11:12:20.423368 30276 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_kobm8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 11:12:20.423472 30276 mini_ranger.cc:193] Using host: 127.29.145.20
I20260811 11:12:20.428819 30276 mini_ranger.cc:227] Ranger admin URL: http://127.29.145.20:53747
Aug 11, 2026 11:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 11:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 11:12:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 11:12:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 11:12:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 11:12:21.700723 30276 mini_ranger.cc:160] Time spent starting Ranger: real 10.999s user 0.004s sys 0.012s
Aug 11, 2026 11:12:43 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 11:12:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 11:12:43 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 11:12:43 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 11:12:44 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 11:12:45.007658 30276 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.29.145.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.29.145.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangerkms_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.29.145.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/rangerkms_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.29.145.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.29.145.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.29.145.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/HTTP_127.29.145.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 11:12:45 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446765, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 11:12:45.114785 30276 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 11:12:45.126026 30276 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 11:12:46.156905 30276 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 11:12:46.157037 30276 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms
I20260811 11:12:46.157068 30276 mini_ranger_kms.cc:227] Using postgres at 127.29.145.20:37809
2026-08-11 11:12:46,374 [I] DB FLAVOR :POSTGRES
2026-08-11 11:12:46,374 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:12:46,374 [I] Checking connection
2026-08-11 11:12:46,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 11:12:46,653 [I] connection success
2026-08-11 11:12:46,654 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 11:12:46,654 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 11:12:46,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 11:12:46,965 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 11:12:46,965 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:12:46,965 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 11:12:46,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.29.145.20:37809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 11:12:47,269 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 11:12:47.735832 30276 mini_ranger_kms.cc:310] Created kms service
I20260811 11:12:47.952675 30276 mini_ranger_kms.cc:326] Created kudu user
I20260811 11:12:47.997233 30276 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 11:12:47 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446765, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:48.435222 30276 mini_ranger_kms.cc:384] Added ranger policy
I20260811 11:12:48.435474 30276 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms:/tmp/dist-test-task_kobm8/build/release/bin/postgresql.jar:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_kobm8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 11:12:48.435503 30276 mini_ranger_kms.cc:240] Using host: 127.29.145.20
I20260811 11:12:48.439986 30276 mini_ranger_kms.cc:276] Ranger KMS PID: 11754
I20260811 11:12:48.440061 30276 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.29.145.20:33243
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 11:12:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 11:12:50 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786446770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:50.869756 30276 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.755s user 0.003s sys 0.195s
I20260811 11:12:50.869993 30276 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 11:12:50.870045 30276 mini_ranger_kms.cc:401] 127.29.145.20:33243/kms/v1/keys
Aug 11 11:12:51 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
Aug 11 11:12:51 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.29.145.20@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
Aug 11 11:12:53 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:12:53.717940 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38625
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:38625
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:53.806279 11822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:53.806490 11822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:53.806512 11822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:53.807670 11822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:12:53.807720 11822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:53.807735 11822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:53.807747 11822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:12:53.807760 11822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:12:53.809406 11822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
--ranger_kms_url=127.29.145.20:33243/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:38625
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:53.809667 11822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:53.809949 11822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:53.815487 11830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:53.815483 11829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:53.815529 11832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:53.815708 11822 server_base.cc:1034] running on GCE node
I20260811 11:12:53.815968 11822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:53.816339 11822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:53.817518 11822 hybrid_clock.cc:648] HybridClock initialized: now 1786446773817491 us; error 43 us; skew 500 ppm
Aug 11 11:12:53 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446773, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:53.820451 11822 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:12:53.820957 11822 webserver.cc:460] Webserver started at http://127.29.145.62:37615/ using document root <none> and password file <none>
I20260811 11:12:53.821221 11822 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:53.821285 11822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:53.821441 11822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:53 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446773, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:53.879210 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "fc1cce0c0bee48a4b2076a616c02063f"
format_stamp: "Formatted at 2026-08-11 11:12:53 on dist-test-slave-69rg"
server_key: "4eefb7fe4f938fbebdf8f6a31a604187"
server_key_iv: "06388469b62d3e62d12a5a729250965b"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:53.879691 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "fc1cce0c0bee48a4b2076a616c02063f"
format_stamp: "Formatted at 2026-08-11 11:12:53 on dist-test-slave-69rg"
server_key: "4eefb7fe4f938fbebdf8f6a31a604187"
server_key_iv: "06388469b62d3e62d12a5a729250965b"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:53.881605 11822 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:53.894670 11839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:53.894923 11822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:12:53.895136 11822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "fc1cce0c0bee48a4b2076a616c02063f"
format_stamp: "Formatted at 2026-08-11 11:12:53 on dist-test-slave-69rg"
server_key: "4eefb7fe4f938fbebdf8f6a31a604187"
server_key_iv: "06388469b62d3e62d12a5a729250965b"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:53.895277 11822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:53.928357 11822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:53.930855 11822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:53.931013 11822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:53.936457 11822 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:38625
I20260811 11:12:53.936488 11896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:38625 every 8 connection(s)
I20260811 11:12:53.937026 11822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:12:53.937871 11898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:53.940182 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Bootstrap starting.
I20260811 11:12:53.940824 11898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:53.941131 11898 log.cc:824] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:53.941933 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: No bootstrap required, opened a new log
I20260811 11:12:53.943967 11898 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:53.944144 11898 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:53.944192 11898 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1cce0c0bee48a4b2076a616c02063f, State: Initialized, Role: FOLLOWER
I20260811 11:12:53.944365 11898 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:53.944446 11898 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:12:53.944490 11898 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:12:53.944538 11898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:53.945092 11898 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:53.945212 11898 leader_election.cc:302] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc1cce0c0bee48a4b2076a616c02063f; no voters:
I20260811 11:12:53.945443 11898 leader_election.cc:288] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:12:53.945482 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 11822
I20260811 11:12:53.945492 11901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:12:53.945592 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:12:53.945717 11901 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 LEADER]: Becoming Leader. State: Replica: fc1cce0c0bee48a4b2076a616c02063f, State: Running, Role: LEADER
I20260811 11:12:53.945797 30276 external_mini_cluster.cc:1428] Setting key 64c59dd465b9a59497d2dc89304a6bad
I20260811 11:12:53.945806 11901 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:53.945835 11898 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:12:53.946224 11901 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc1cce0c0bee48a4b2076a616c02063f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } } }
I20260811 11:12:53.946377 11901 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:53.946573 11901 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc1cce0c0bee48a4b2076a616c02063f. Latest consensus state: current_term: 1 leader_uuid: "fc1cce0c0bee48a4b2076a616c02063f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } } }
I20260811 11:12:53.946695 11901 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:53.946851 11909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:12:53.947399 11909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:12:53.947669 11898 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:12:53.956117 11909 catalog_manager.cc:1347] Generated new cluster ID: cdf1df33631240b386692975d753a120
I20260811 11:12:53.956218 11909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:12:53 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:53.962994 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:53.946439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.29.145.62:38625)
0811 11:12:53.946891 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:53.946902 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:53.946934 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:12:53.947059 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:53.947067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:53.947131 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:53.947286 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:53.948249 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:53.952099 (+ 3850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:53.953297 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:53.954206 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:53.954368 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:53.957589 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:53.957594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:53.957594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:53.957607 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:53.960656 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:53.961078 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:53.961082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:53.961082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:53.961181 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:53.961370 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:53.961371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:53.961371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:53.961693 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:53.962403 (+ 710us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:53.962490 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 11:12:53.962646 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":79,"threads_started":1}
I20260811 11:12:53.993858 11909 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:12:53.994747 11909 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:12:54.003105 11909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Generated new TSK 0
I20260811 11:12:54.003378 11909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:12:54.733688 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:54.860036 11952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:54.860343 11952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:54.860390 11952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:54.862182 11952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:54.862243 11952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:54.862284 11952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:12:54.864707 11952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:54.864975 11952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:54.865289 11952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:54.871850 11958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:54.871851 11960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:54.872047 11957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:54.872323 11952 server_base.cc:1034] running on GCE node
I20260811 11:12:54.872535 11952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:54.872790 11952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:54.873937 11952 hybrid_clock.cc:648] HybridClock initialized: now 1786446774873916 us; error 35 us; skew 500 ppm
Aug 11 11:12:54 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446774, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:54.877409 11952 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:12:54.878050 11952 webserver.cc:460] Webserver started at http://127.29.145.1:32815/ using document root <none> and password file <none>
I20260811 11:12:54.878381 11952 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:54.878474 11952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:54.878633 11952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:54 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446774, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:54.897576 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "ebec9344fe234ad380c2cdd672a06d1e"
format_stamp: "Formatted at 2026-08-11 11:12:54 on dist-test-slave-69rg"
server_key: "0c42f720888ee36eae01c1f9a6079707"
server_key_iv: "8f2b7d51e49056119d54aea7256d4d96"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:54.898236 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "ebec9344fe234ad380c2cdd672a06d1e"
format_stamp: "Formatted at 2026-08-11 11:12:54 on dist-test-slave-69rg"
server_key: "0c42f720888ee36eae01c1f9a6079707"
server_key_iv: "8f2b7d51e49056119d54aea7256d4d96"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:54.900290 11952 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:12:54.926491 11970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:54.926750 11952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:12:54.926851 11952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "ebec9344fe234ad380c2cdd672a06d1e"
format_stamp: "Formatted at 2026-08-11 11:12:54 on dist-test-slave-69rg"
server_key: "0c42f720888ee36eae01c1f9a6079707"
server_key_iv: "8f2b7d51e49056119d54aea7256d4d96"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:54.926949 11952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:54.944195 11952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:54.946566 11952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:54.946739 11952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:54.947006 11952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:54.947458 11952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:54.947516 11952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:54.947587 11952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:54.947631 11952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:54.955185 11952 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44847
I20260811 11:12:54.955610 11952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:12:54.958359 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44847 every 8 connection(s)
I20260811 11:12:54.960546 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 11952
I20260811 11:12:54.960676 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:12:54.960875 30276 external_mini_cluster.cc:1428] Setting key 2668dd0aa2a4c944842bebd38c2dbd2d
Aug 11 11:12:54 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446774, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:54.981426 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:54.959066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.1:44269)
0811 11:12:54.959267 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:54.959311 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:54.959828 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:54.962252 (+ 2424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:54.962257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:54.962527 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:54.962831 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:54.962838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:54.972494 (+ 9656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:54.972497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:54.973989 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:54.973994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:54.974695 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:54.974696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:54.974853 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:54.975557 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:54.975570 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:54.977310 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:54.979633 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:54.979642 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:54.979645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:54.979973 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:54.980205 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:54.980206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:54.980206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:54.980342 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:54.980636 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:54.980642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:54.980830 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 11:12:54.980944 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 11:12:54.981111 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260811 11:12:54.982120 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:54.959097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:44269 (local address 127.29.145.62:38625)
0811 11:12:54.961941 (+ 2844us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:54.961944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:54.961957 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:12:54.961988 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:54.961989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:54.962003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:54.962944 (+ 941us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:54.962992 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:54.972309 (+ 9317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:54.974127 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:54.974459 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:54.974553 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:54.977434 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:54.977435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:54.977436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:54.977444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:54.979523 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:54.980065 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:54.980066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:54.980067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:54.980135 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:54.980413 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:54.980414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:54.980414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:54.980555 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:54.980602 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:54.981923 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0811 11:12:54.982021 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2802,"thread_start_us":124,"threads_started":1}
I20260811 11:12:54.982772 12100 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:54.982878 12100 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:54.983072 12100 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:54.983641 11857 ts_manager.cc:194] Registered new tserver with Master: ebec9344fe234ad380c2cdd672a06d1e (127.29.145.1:44847)
I20260811 11:12:54.984593 11857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:44269
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:12:55.024950 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 11:12:55.148270 12109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:55.148554 12109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:55.148598 12109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:55.150276 12109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:55.150331 12109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:55.150368 12109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:12:55.152525 12109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:55.152798 12109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:55.153107 12109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:55.159799 12122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.159817 12119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.160241 12120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:55.160549 12109 server_base.cc:1034] running on GCE node
I20260811 11:12:55.160758 12109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:55.161043 12109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:55.162221 12109 hybrid_clock.cc:648] HybridClock initialized: now 1786446775162181 us; error 47 us; skew 500 ppm
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:55.165454 12109 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:12:55.166018 12109 webserver.cc:460] Webserver started at http://127.29.145.2:39547/ using document root <none> and password file <none>
I20260811 11:12:55.166319 12109 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:55.166399 12109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:55.166571 12109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:55.181602 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "90863b47479c4f0d8d6fd62d355e2a5f"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "991904f8b4d0609b2707f51b3f834a0b"
server_key_iv: "fcdc73b26843c2ccb436cebc33045565"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.182232 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "90863b47479c4f0d8d6fd62d355e2a5f"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "991904f8b4d0609b2707f51b3f834a0b"
server_key_iv: "fcdc73b26843c2ccb436cebc33045565"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.184234 12109 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:55.209986 12129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.210361 12109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:55.210536 12109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "90863b47479c4f0d8d6fd62d355e2a5f"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "991904f8b4d0609b2707f51b3f834a0b"
server_key_iv: "fcdc73b26843c2ccb436cebc33045565"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.210752 12109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:55.242545 12109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:55.245165 12109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:55.245406 12109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:55.245805 12109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:55.246315 12109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:55.246418 12109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.246500 12109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:55.246568 12109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.254998 12109 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:43533
I20260811 11:12:55.255560 12109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:12:55.256727 12264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:43533 every 8 connection(s)
I20260811 11:12:55.262084 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12109
I20260811 11:12:55.262194 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:12:55.262401 30276 external_mini_cluster.cc:1428] Setting key b3332ed29efa4ab10d2ddf3115a96021
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:55.270404 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:55.256948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:59341 (local address 127.29.145.62:38625)
0811 11:12:55.256999 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:55.257001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:55.257094 (+ 93us) server_negotiation.cc:407] Connection header received
0811 11:12:55.257883 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:55.257885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:55.257904 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:55.257993 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:55.258836 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.259678 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.261222 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.261579 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.261624 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:55.264597 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:55.264598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:55.264599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:55.264606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:55.266504 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:55.269079 (+ 2575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:55.269080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:55.269081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:55.269154 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:55.269490 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:55.269492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:55.269492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:55.269667 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:55.269718 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:55.270182 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 11:12:55.270284 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:12:55.271221 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:55.256652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.2:59341)
0811 11:12:55.256952 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:55.256999 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:55.257726 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:55.258090 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:55.258096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:55.258368 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:55.258728 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:55.258735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.259798 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.259799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:55.261111 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:55.261114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.261737 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.261738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:55.261911 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:55.262543 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:55.262554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:55.264484 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:55.266632 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:55.266640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:55.266642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:55.266951 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:55.269246 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:55.269247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:55.269247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:55.269400 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:55.269778 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:55.269785 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:55.270005 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 11:12:55.270769 (+ 764us) client_negotiation.cc:240] Negotiation successful
0811 11:12:55.270964 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":84,"threads_started":1}
I20260811 11:12:55.272038 12265 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:55.272138 12265 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:55.272353 12265 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:55.272946 11857 ts_manager.cc:194] Registered new tserver with Master: 90863b47479c4f0d8d6fd62d355e2a5f (127.29.145.2:43533)
I20260811 11:12:55.273440 11857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:59341
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:12:55.301782 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:55.419783 12270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:55.420068 12270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:55.420109 12270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:55.421787 12270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:55.421887 12270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:55.421928 12270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:12:55.424196 12270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:55.424464 12270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:55.424759 12270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:55.431716 12276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.431802 12275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.432042 12278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:55.433193 12270 server_base.cc:1034] running on GCE node
I20260811 11:12:55.433436 12270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:55.433733 12270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:55.434916 12270 hybrid_clock.cc:648] HybridClock initialized: now 1786446775434885 us; error 47 us; skew 500 ppm
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:55.438552 12270 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:12:55.439267 12270 webserver.cc:460] Webserver started at http://127.29.145.3:39381/ using document root <none> and password file <none>
I20260811 11:12:55.439621 12270 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:55.439759 12270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:12:55.439951 12270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:55.454361 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "a81ce2cb162f4474b74f942a18579c34"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "8ba2f5a8601b39c0142b0ac186b9cbe9"
server_key_iv: "b5f57e9bc7cc736a527d83e16f757bea"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.454932 12270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "a81ce2cb162f4474b74f942a18579c34"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "8ba2f5a8601b39c0142b0ac186b9cbe9"
server_key_iv: "b5f57e9bc7cc736a527d83e16f757bea"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.457556 12270 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:12:55.478101 12287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.478890 12270 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.001s
I20260811 11:12:55.479012 12270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "a81ce2cb162f4474b74f942a18579c34"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "8ba2f5a8601b39c0142b0ac186b9cbe9"
server_key_iv: "b5f57e9bc7cc736a527d83e16f757bea"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.479125 12270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:55.503722 12270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:55.506321 12270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:55.506532 12270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:55.506788 12270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:55.507203 12270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:55.507289 12270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.507368 12270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:55.507427 12270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.515805 12270 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:44485
I20260811 11:12:55.516376 12270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:12:55.518996 12425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:44485 every 8 connection(s)
I20260811 11:12:55.524364 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12270
I20260811 11:12:55.524470 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:12:55.524641 30276 external_mini_cluster.cc:1428] Setting key a188df824a3113ea3e0120ebac93e1c3
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:55.531802 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:55.518802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:49329 (local address 127.29.145.62:38625)
0811 11:12:55.518848 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:55.518850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:55.519574 (+ 724us) server_negotiation.cc:407] Connection header received
0811 11:12:55.520299 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:55.520300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:55.520313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:55.520345 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:55.521222 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.521999 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.523438 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.523723 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.523771 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:55.528417 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:55.528419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:55.528419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:55.528428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:55.530255 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:55.530733 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:55.530735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:55.530735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:55.530793 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:55.531040 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:55.531041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:55.531041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:55.531179 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:55.531221 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:55.531605 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 11:12:55.531688 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:12:55.532325 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:55.518511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.3:49329)
0811 11:12:55.519446 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:55.519487 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:55.520157 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:55.520533 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:55.520537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:55.520817 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:55.521099 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:55.521106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.522110 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.522111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:55.523338 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:55.523341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:55.523878 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:55.523879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:55.524018 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:55.526044 (+ 2026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:55.526055 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:55.528303 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:55.530360 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:55.530367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:55.530369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:55.530642 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:55.530859 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:55.530860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:55.530860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:55.530975 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:55.531266 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:55.531272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:55.531448 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 11:12:55.531932 (+ 484us) client_negotiation.cc:240] Negotiation successful
0811 11:12:55.532089 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":92,"threads_started":1}
I20260811 11:12:55.532764 12426 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:55.532848 12426 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:55.533026 12426 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:55.533350 11857 ts_manager.cc:194] Registered new tserver with Master: a81ce2cb162f4474b74f942a18579c34 (127.29.145.3:44485)
I20260811 11:12:55.533799 11857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:49329
I20260811 11:12:55.536356 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:12:55.537225 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 11952
I20260811 11:12:55.546926 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12109
I20260811 11:12:55.562064 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12270
I20260811 11:12:55.570118 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 11822
I20260811 11:12:55.581866 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38625
--webserver_interface=127.29.145.62
--webserver_port=37615
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:38625
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:12:55.671085 12430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:55.671293 12430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:55.671314 12430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:55.672374 12430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:12:55.672408 12430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:55.672422 12430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:55.672434 12430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:12:55.672446 12430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:12:55.673962 12430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:38625
--ranger_config_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=37615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:55.674150 12430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:55.674309 12430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:55.679335 12437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.679366 12436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.679340 12439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:55.679656 12430 server_base.cc:1034] running on GCE node
I20260811 11:12:55.679929 12430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:55.680255 12430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:55.681448 12430 hybrid_clock.cc:648] HybridClock initialized: now 1786446775681428 us; error 43 us; skew 500 ppm
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:55.684118 12430 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:12:55.684616 12430 webserver.cc:460] Webserver started at http://127.29.145.62:37615/ using document root <none> and password file <none>
I20260811 11:12:55.684813 12430 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:55.684863 12430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:55.769408 12430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:12:55.770248 12449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.770442 12430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:12:55.770567 12430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "fc1cce0c0bee48a4b2076a616c02063f"
format_stamp: "Formatted at 2026-08-11 11:12:53 on dist-test-slave-69rg"
server_key: "4eefb7fe4f938fbebdf8f6a31a604187"
server_key_iv: "06388469b62d3e62d12a5a729250965b"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.770954 12430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:55.780730 12430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:55.782810 12430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:55.782976 12430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:55.787775 12430 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:38625
I20260811 11:12:55.787890 12512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:38625 every 8 connection(s)
I20260811 11:12:55.788267 12430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:12:55.789610 12513 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:12:55.790288 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12430
I20260811 11:12:55.790537 12513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Bootstrap starting.
I20260811 11:12:55.790586 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:44847
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:38625
--webserver_port=32815
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:12:55.798628 12513 log.cc:824] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:55.800377 12513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:12:55.800683 12513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Bootstrap complete.
I20260811 11:12:55.803174 12513 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:55.803437 12513 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc1cce0c0bee48a4b2076a616c02063f, State: Initialized, Role: FOLLOWER
I20260811 11:12:55.803629 12513 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:55.803714 12513 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:12:55.803768 12513 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:12:55.803818 12513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:12:55.804796 12513 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:55.804919 12513 leader_election.cc:302] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc1cce0c0bee48a4b2076a616c02063f; no voters:
I20260811 11:12:55.805122 12513 leader_election.cc:288] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:12:55.805160 12517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:12:55.805284 12517 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [term 2 LEADER]: Becoming Leader. State: Replica: fc1cce0c0bee48a4b2076a616c02063f, State: Running, Role: LEADER
I20260811 11:12:55.805418 12513 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:12:55.805481 12517 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } }
I20260811 11:12:55.805860 12518 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fc1cce0c0bee48a4b2076a616c02063f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } } }
I20260811 11:12:55.805881 12520 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc1cce0c0bee48a4b2076a616c02063f. Latest consensus state: current_term: 2 leader_uuid: "fc1cce0c0bee48a4b2076a616c02063f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc1cce0c0bee48a4b2076a616c02063f" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38625 } } }
I20260811 11:12:55.806171 12518 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:55.806182 12520 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f [sys.catalog]: This master's current role is: LEADER
I20260811 11:12:55.806998 12513 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:12:55.809478 12529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:12:55.810606 12529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:12:55.810920 12529 catalog_manager.cc:1259] Loaded cluster ID: cdf1df33631240b386692975d753a120
I20260811 11:12:55.811023 12529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:12:55.812629 12529 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:12:55.812906 12529 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Loaded TSK: 0
I20260811 11:12:55.813271 12529 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:12:55.905969 12515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:55.906347 12515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:55.906405 12515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:55.908100 12515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:55.908172 12515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:55.908218 12515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:12:55.910465 12515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:44847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=32815
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:55.910859 12515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:55.911146 12515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:55.917757 12558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.917780 12557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:55.917757 12562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:55.919068 12515 server_base.cc:1034] running on GCE node
I20260811 11:12:55.919299 12515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:55.919575 12515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:55.920761 12515 hybrid_clock.cc:648] HybridClock initialized: now 1786446775920741 us; error 40 us; skew 500 ppm
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:55.928555 12515 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:12:55.929180 12515 webserver.cc:460] Webserver started at http://127.29.145.1:32815/ using document root <none> and password file <none>
I20260811 11:12:55.929477 12515 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:55.929574 12515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:12:55 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:55.950292 12515 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.001s
I20260811 11:12:55.951545 12571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:55.951810 12515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:12:55.951952 12515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "ebec9344fe234ad380c2cdd672a06d1e"
format_stamp: "Formatted at 2026-08-11 11:12:54 on dist-test-slave-69rg"
server_key: "0c42f720888ee36eae01c1f9a6079707"
server_key_iv: "8f2b7d51e49056119d54aea7256d4d96"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:55.952486 12515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:55.994944 12515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:56.001060 12515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:56.001296 12515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:56.001606 12515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:56.002063 12515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:56.002169 12515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.002270 12515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:56.002354 12515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.009594 12515 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44847
I20260811 11:12:56.009671 12698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44847 every 8 connection(s)
I20260811 11:12:56.010212 12515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446775, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:56.020292 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12515
I20260811 11:12:56.020602 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:43533
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:38625
--webserver_port=39547
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:12:56.024987 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.011235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.1:51187)
0811 11:12:56.011444 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:56.011484 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:56.012128 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:56.012633 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:56.012637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:56.012850 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:56.013045 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.013050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.014290 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.014292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.015084 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.015087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.015719 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.015720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.015845 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.016429 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:56.016438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:56.018049 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:56.022997 (+ 4948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.023004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.023006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.023322 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.023628 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.023629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.023629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.023763 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.024289 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:56.024297 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:56.024450 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 11:12:56.024565 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 11:12:56.024745 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260811 11:12:56.025501 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.011516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:51187 (local address 127.29.145.62:38625)
0811 11:12:56.011756 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:56.011776 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:56.011821 (+ 45us) server_negotiation.cc:407] Connection header received
0811 11:12:56.012377 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:56.012384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:56.012444 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:56.012590 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:56.013180 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.014180 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.015197 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.015604 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.015731 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.018202 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:56.018207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:56.018207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:56.018219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:56.022891 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.023428 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.023431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.023432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.023539 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.023844 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.023845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.023846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.024172 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:56.024256 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:56.025174 (+ 918us) server_negotiation.cc:299] Negotiation successful
0811 11:12:56.025306 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
I20260811 11:12:56.026484 12699 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:56.026587 12699 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.026785 12699 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.027899 12699 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 11:12:56.027988 12699 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.028066 12699 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:56.028424 12699 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.028528 12699 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.028849 12699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:12:56.130669 12703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:56.130892 12703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:56.130931 12703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:56.132308 12703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:56.132411 12703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:56.132726 12703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:12:56.134711 12703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:43533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=39547
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:56.134994 12703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:56.135236 12703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:56.141538 12711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:56.141682 12710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:56.141546 12713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:56.142184 12703 server_base.cc:1034] running on GCE node
I20260811 11:12:56.142401 12703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:56.142658 12703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:56.143824 12703 hybrid_clock.cc:648] HybridClock initialized: now 1786446776143800 us; error 39 us; skew 500 ppm
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:56.146940 12703 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:12:56.147467 12703 webserver.cc:460] Webserver started at http://127.29.145.2:39547/ using document root <none> and password file <none>
I20260811 11:12:56.147725 12703 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:56.147822 12703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:56.166503 12703 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260811 11:12:56.167537 12721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.167723 12703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:12:56.167846 12703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "90863b47479c4f0d8d6fd62d355e2a5f"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "991904f8b4d0609b2707f51b3f834a0b"
server_key_iv: "fcdc73b26843c2ccb436cebc33045565"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:56.168298 12703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:56.192281 12703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:56.194548 12703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:56.194706 12703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:56.194978 12703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:56.195322 12703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:56.195379 12703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.195420 12703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:56.195447 12703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.202224 12703 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:43533
I20260811 11:12:56.202337 12854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:43533 every 8 connection(s)
I20260811 11:12:56.202672 12703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:12:56.208798 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12703
I20260811 11:12:56.209123 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.29.145.20:33243/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:44485
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:38625
--webserver_port=39381
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:56.213585 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.203630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:41497 (local address 127.29.145.62:38625)
0811 11:12:56.203680 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:56.203682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:56.203962 (+ 280us) server_negotiation.cc:407] Connection header received
0811 11:12:56.204529 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:56.204531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:56.204547 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:56.204604 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:56.205278 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.205793 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.207023 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.207379 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.207458 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.209935 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:56.209937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:56.209937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:56.209946 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:56.211809 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.212384 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.212385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.212386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.212450 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.212731 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.212732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.212732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.212877 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:56.212924 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:56.213341 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 11:12:56.213445 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:12:56.214479 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.203640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.2:41497)
0811 11:12:56.203806 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:56.203848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:56.204411 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:56.204674 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:56.204679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:56.204871 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:56.205183 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.205189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.205934 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.205935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.206920 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.206923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.207485 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.207487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.207624 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.208178 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:56.208187 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:56.209791 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:56.211915 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.211923 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.211926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.212286 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.212522 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.212523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.212523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.212657 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.212976 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:56.212982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:56.213155 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 11:12:56.213990 (+ 835us) client_negotiation.cc:240] Negotiation successful
0811 11:12:56.214181 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260811 11:12:56.215245 12855 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:56.215348 12855 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.215549 12855 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.216226 12855 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 11:12:56.216320 12855 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.216398 12855 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:56.217013 12855 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.217092 12855 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.217443 12855 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:12:56.324170 12858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:12:56.324471 12858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:12:56.324512 12858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:12:56.326213 12858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:12:56.326328 12858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:12:56.326408 12858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:12:56.328639 12858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.29.145.20:33243/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:44485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=39381
--tserver_master_addrs=127.29.145.62:38625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:12:56.329116 12858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:12:56.329447 12858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:12:56.335150 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:12:56.335577 12870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:56.335909 12858 server_base.cc:1034] running on GCE node
W20260811 11:12:56.336637 12869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:12:56.336915 12858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:12:56.337283 12858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:12:56.338526 12858 hybrid_clock.cc:648] HybridClock initialized: now 1786446776338502 us; error 36 us; skew 500 ppm
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:56.341935 12858 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:12:56.342530 12858 webserver.cc:460] Webserver started at http://127.29.145.3:39381/ using document root <none> and password file <none>
I20260811 11:12:56.342832 12858 fs_manager.cc:359] Metadata directory not provided
I20260811 11:12:56.342921 12858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:56.363149 12858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:12:56.364114 12881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.364392 12858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:12:56.364492 12858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "a81ce2cb162f4474b74f942a18579c34"
format_stamp: "Formatted at 2026-08-11 11:12:55 on dist-test-slave-69rg"
server_key: "8ba2f5a8601b39c0142b0ac186b9cbe9"
server_key_iv: "b5f57e9bc7cc736a527d83e16f757bea"
server_key_version: "kuduclusterkey@0"
I20260811 11:12:56.365025 12858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:12:56.376646 12858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:12:56.378808 12858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:12:56.378960 12858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:12:56.379209 12858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:12:56.379595 12858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:12:56.379647 12858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.379676 12858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:12:56.379694 12858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:12:56.387316 12858 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:44485
I20260811 11:12:56.387856 12858 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:12:56.389359 13009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:44485 every 8 connection(s)
I20260811 11:12:56.390981 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 12858
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:12:56.401351 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.388971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38625 (local address 127.29.145.3:57001)
0811 11:12:56.389473 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:56.389519 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:56.390261 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:56.392500 (+ 2239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:56.392506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:56.392705 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:12:56.392909 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.392913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.393707 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.393709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.394603 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:56.394606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.395180 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.395182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:56.395341 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.395967 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:12:56.395976 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:12:56.397416 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:12:56.399370 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.399378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.399381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.399675 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.399994 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:12:56.399995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:12:56.399996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:12:56.400114 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:12:56.400535 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:12:56.400543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:12:56.400749 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 11:12:56.400877 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 11:12:56.401057 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":87,"threads_started":1}
I20260811 11:12:56.401899 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.389211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:57001 (local address 127.29.145.62:38625)
0811 11:12:56.389261 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:56.389264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:56.389613 (+ 349us) server_negotiation.cc:407] Connection header received
0811 11:12:56.392351 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:56.392353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:56.392367 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:56.392416 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:56.393013 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.393591 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.394704 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.395010 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.395082 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.397517 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:56.397519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:56.397519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:56.397527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:56.399275 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.399779 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.399781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.399782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.399862 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.400233 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.400235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.400235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.400429 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:56.400503 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:56.401659 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0811 11:12:56.401749 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:12:56.402604 13010 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38625
I20260811 11:12:56.402738 13010 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.402971 13010 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.403690 13010 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 11:12:56.403775 13010 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.403853 13010 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:56.404167 13010 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:56.404246 13010 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
W20260811 11:12:56.404525 13010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:12:56.405884 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:56.391450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.29.145.62:38625)
0811 11:12:56.391584 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:56.391586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:56.393695 (+ 2109us) server_negotiation.cc:407] Connection header received
0811 11:12:56.393717 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:56.393717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:56.393730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:56.393796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:56.395610 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.397080 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.399378 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:56.399678 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:56.399749 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:56.401862 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:56.401863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:56.401864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:56.401872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:56.404694 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.405050 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.405051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.405051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.405115 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:56.405251 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:56.405252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:56.405252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:56.405372 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:56.405557 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:56.405662 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 11:12:56.405755 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
Aug 11 11:12:56 dist-test-slave-69rg krb5kdc[11291](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:12:57.029007 12699 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:57.029206 12699 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:57.029709 12467 ts_manager.cc:194] Registered new tserver with Master: ebec9344fe234ad380c2cdd672a06d1e (127.29.145.1:44847)
I20260811 11:12:57.030563 12467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:51187
I20260811 11:12:57.217684 12855 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:57.217895 12855 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:57.218386 12467 ts_manager.cc:194] Registered new tserver with Master: 90863b47479c4f0d8d6fd62d355e2a5f (127.29.145.2:43533)
I20260811 11:12:57.218910 12467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:41497
Aug 11 11:12:57 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:57.404752 13010 heartbeater.cc:464] Registering TS with master...
I20260811 11:12:57.404968 13010 heartbeater.cc:511] Master 127.29.145.62:38625 requested a full tablet report, sending...
I20260811 11:12:57.405341 12467 ts_manager.cc:194] Registered new tserver with Master: a81ce2cb162f4474b74f942a18579c34 (127.29.145.3:44485)
I20260811 11:12:57.405928 12467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:57001
I20260811 11:12:57.413127 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:12:57.422492 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.414231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.29.145.62:38625)
0811 11:12:57.414558 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.414560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.414568 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:12:57.414591 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.414592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.414606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.414685 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:12:57.415406 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.416189 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.417235 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.417542 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.417624 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.419160 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:12:57.419162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:12:57.419162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:12:57.419170 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:12:57.421026 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:57.421435 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:57.421437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:57.421437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:57.421508 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:12:57.421863 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:12:57.421864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:12:57.421865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:12:57.422016 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:12:57.422128 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.422292 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.422386 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":81,"threads_started":1}
I20260811 11:12:57.424839 12467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:12:57.427330 12467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:12:57.436806 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.430595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:43533 (local address 127.0.0.1:54198)
0811 11:12:57.431598 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.431744 (+ 146us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:57.431833 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:57.432166 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:57.432170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:57.432173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:57.432243 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.432245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.433792 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.433793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.435297 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.435300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.436468 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.436469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.436559 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.436574 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:12:57.436648 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:12:57.436707 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":42,"thread_start_us":107,"threads_started":1}
I20260811 11:12:57.436995 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.431316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50830 (local address 127.29.145.1:44847)
0811 11:12:57.431448 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.431452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.432083 (+ 631us) server_negotiation.cc:407] Connection header received
0811 11:12:57.432122 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.432125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.432141 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.432210 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:57.432434 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.433779 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.435337 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.436459 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.436540 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.436602 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.436872 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.436914 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260811 11:12:57.437371 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.431125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.29.145.2:43533)
0811 11:12:57.431271 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.431277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.431821 (+ 544us) server_negotiation.cc:407] Connection header received
0811 11:12:57.431922 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.431924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.431942 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.432026 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:57.432326 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.433666 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.435412 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.436273 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.436363 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.436426 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.437234 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.437287 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260811 11:12:57.437196 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.430599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44847 (local address 127.0.0.1:50830)
0811 11:12:57.431967 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.431976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:57.432024 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:57.432298 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:57.432299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:57.432302 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:57.432347 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.432349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.433914 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.433915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.435231 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.435234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.436642 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.436643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.436704 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.436708 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:12:57.436762 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:12:57.436818 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":42,"thread_start_us":108,"threads_started":1}
I20260811 11:12:57.438352 12777 tablet_service.cc:1467] Processing CreateTablet for tablet 131229eca714441fbbab07e7f3811fb3 (DEFAULT_TABLE table=test-table [id=5dd1671065464ab78791b03d2ea65693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:57.438761 12777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 131229eca714441fbbab07e7f3811fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:57.441555 13046 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Bootstrap starting.
I20260811 11:12:57.442094 12626 tablet_service.cc:1467] Processing CreateTablet for tablet 131229eca714441fbbab07e7f3811fb3 (DEFAULT_TABLE table=test-table [id=5dd1671065464ab78791b03d2ea65693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:57.442420 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.430883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:44485 (local address 127.0.0.1:47940)
0811 11:12:57.437626 (+ 6743us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.437637 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:57.437675 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:57.438354 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:57.438356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:57.438358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:57.438394 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.438396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.439830 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.439831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.441043 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.441045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.442187 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.442188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.442260 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.442264 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:12:57.442296 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 11:12:57.442339 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6728,"thread_start_us":86,"threads_started":1}
I20260811 11:12:57.442448 12626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 131229eca714441fbbab07e7f3811fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:57.442677 13046 tablet_bootstrap.cc:654] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:57.442781 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.431007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.29.145.3:44485)
0811 11:12:57.438158 (+ 7151us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.438164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.438178 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:12:57.438217 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.438220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.438238 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.438468 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:57.438554 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.439708 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.441169 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.442065 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.442540 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.442621 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.442652 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.442696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7100,"thread_start_us":67,"threads_started":1}
I20260811 11:12:57.443022 13046 log.cc:824] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:57.443445 12912 tablet_service.cc:1467] Processing CreateTablet for tablet 131229eca714441fbbab07e7f3811fb3 (DEFAULT_TABLE table=test-table [id=5dd1671065464ab78791b03d2ea65693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:12:57.443840 12912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 131229eca714441fbbab07e7f3811fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:12:57.443985 13046 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: No bootstrap required, opened a new log
I20260811 11:12:57.444049 13046 ts_tablet_manager.cc:1397] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 11:12:57.447039 13046 raft_consensus.cc:348] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.447299 13046 raft_consensus.cc:374] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:57.447376 13046 raft_consensus.cc:729] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90863b47479c4f0d8d6fd62d355e2a5f, State: Initialized, Role: FOLLOWER
I20260811 11:12:57.447546 13046 consensus_queue.cc:260] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.447966 13049 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Bootstrap starting.
I20260811 11:12:57.447996 13046 ts_tablet_manager.cc:1428] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260811 11:12:57.448140 12855 heartbeater.cc:503] Master 127.29.145.62:38625 was elected leader, sending a full tablet report...
I20260811 11:12:57.448963 13049 tablet_bootstrap.cc:654] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:57.449313 13049 log.cc:824] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:57.450321 13049 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: No bootstrap required, opened a new log
I20260811 11:12:57.450428 13049 ts_tablet_manager.cc:1397] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:12:57.452260 13049 raft_consensus.cc:348] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.452428 13049 raft_consensus.cc:374] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:57.452760 13049 raft_consensus.cc:729] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a81ce2cb162f4474b74f942a18579c34, State: Initialized, Role: FOLLOWER
I20260811 11:12:57.452989 13049 consensus_queue.cc:260] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.453351 13010 heartbeater.cc:503] Master 127.29.145.62:38625 was elected leader, sending a full tablet report...
I20260811 11:12:57.453336 13049 ts_tablet_manager.cc:1428] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260811 11:12:57.454181 12856 tablet.cc:2369] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:12:57.454417 13048 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Bootstrap starting.
I20260811 11:12:57.455044 13048 tablet_bootstrap.cc:654] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Neither blocks nor log segments found. Creating new log.
I20260811 11:12:57.455425 13048 log.cc:824] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:12:57.456357 13048 tablet_bootstrap.cc:492] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: No bootstrap required, opened a new log
I20260811 11:12:57.456449 13048 ts_tablet_manager.cc:1397] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:12:57.458514 13048 raft_consensus.cc:348] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.458683 13048 raft_consensus.cc:374] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:12:57.458765 13048 raft_consensus.cc:729] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebec9344fe234ad380c2cdd672a06d1e, State: Initialized, Role: FOLLOWER
I20260811 11:12:57.459074 13048 consensus_queue.cc:260] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.459468 12699 heartbeater.cc:503] Master 127.29.145.62:38625 was elected leader, sending a full tablet report...
I20260811 11:12:57.459470 13048 ts_tablet_manager.cc:1428] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260811 11:12:57.511884 12700 tablet.cc:2369] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:12:57.512408 13053 raft_consensus.cc:482] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:12:57.512543 13053 raft_consensus.cc:504] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.512976 13053 leader_election.cc:288] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebec9344fe234ad380c2cdd672a06d1e (127.29.145.1:44847), 90863b47479c4f0d8d6fd62d355e2a5f (127.29.145.2:43533)
I20260811 11:12:57.517920 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.513499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:35205 (local address 127.29.145.2:43533)
0811 11:12:57.513547 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.513549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.513789 (+ 240us) server_negotiation.cc:407] Connection header received
0811 11:12:57.513848 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.513849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.513860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.513917 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:57.514263 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.515001 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.516691 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.517397 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.517460 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.517473 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.517768 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.517837 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:12:57.518590 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.513226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:43533 (local address 127.29.145.3:35205)
0811 11:12:57.513578 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.513589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:57.513655 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:57.514024 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:57.514025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:57.514028 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:57.514173 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.514175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.515110 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.515111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.516556 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.516560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.517567 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.517569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.517666 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.517672 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:12:57.518425 (+ 753us) client_negotiation.cc:240] Negotiation successful
0811 11:12:57.518479 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":13,"thread_start_us":127,"threads_started":1}
I20260811 11:12:57.519172 12800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "131229eca714441fbbab07e7f3811fb3" candidate_uuid: "a81ce2cb162f4474b74f942a18579c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" is_pre_election: true
I20260811 11:12:57.519378 12800 raft_consensus.cc:2454] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a81ce2cb162f4474b74f942a18579c34 in term 0.
I20260811 11:12:57.519671 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.513482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:55315 (local address 127.29.145.1:44847)
0811 11:12:57.513547 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.513550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.513783 (+ 233us) server_negotiation.cc:407] Connection header received
0811 11:12:57.513927 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.513928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.513938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.513969 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:12:57.514247 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.515001 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.517899 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.518568 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.518623 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.518632 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.519535 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.519594 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 11:12:57.519670 12882 leader_election.cc:302] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90863b47479c4f0d8d6fd62d355e2a5f, a81ce2cb162f4474b74f942a18579c34; no voters:
I20260811 11:12:57.519824 13053 raft_consensus.cc:2781] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:12:57.519915 13053 raft_consensus.cc:482] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:12:57.519943 13053 raft_consensus.cc:3037] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:57.520073 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.513226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44847 (local address 127.29.145.3:55315)
0811 11:12:57.513734 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.513743 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:12:57.513840 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:12:57.514033 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:12:57.514033 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:12:57.514034 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:12:57.514158 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.514160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.515111 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.515112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.516556 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:12:57.516560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.518700 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.518702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:12:57.518775 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.518777 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:12:57.519920 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0811 11:12:57.519972 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":198,"thread_start_us":114,"threads_started":1}
I20260811 11:12:57.520644 12648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "131229eca714441fbbab07e7f3811fb3" candidate_uuid: "a81ce2cb162f4474b74f942a18579c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebec9344fe234ad380c2cdd672a06d1e" is_pre_election: true
I20260811 11:12:57.520818 12648 raft_consensus.cc:2454] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a81ce2cb162f4474b74f942a18579c34 in term 0.
I20260811 11:12:57.521121 13053 raft_consensus.cc:504] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.521291 13053 leader_election.cc:288] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [CANDIDATE]: Term 1 election: Requested vote from peers ebec9344fe234ad380c2cdd672a06d1e (127.29.145.1:44847), 90863b47479c4f0d8d6fd62d355e2a5f (127.29.145.2:43533)
I20260811 11:12:57.521499 12800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "131229eca714441fbbab07e7f3811fb3" candidate_uuid: "a81ce2cb162f4474b74f942a18579c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90863b47479c4f0d8d6fd62d355e2a5f"
I20260811 11:12:57.521582 12800 raft_consensus.cc:3037] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:57.522033 12648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "131229eca714441fbbab07e7f3811fb3" candidate_uuid: "a81ce2cb162f4474b74f942a18579c34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebec9344fe234ad380c2cdd672a06d1e"
I20260811 11:12:57.522110 12648 raft_consensus.cc:3037] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:12:57.523228 12648 raft_consensus.cc:2454] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a81ce2cb162f4474b74f942a18579c34 in term 1.
I20260811 11:12:57.523890 12884 leader_election.cc:302] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a81ce2cb162f4474b74f942a18579c34, ebec9344fe234ad380c2cdd672a06d1e; no voters:
I20260811 11:12:57.525569 13053 raft_consensus.cc:2781] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:12:57.525759 13053 raft_consensus.cc:686] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 1 LEADER]: Becoming Leader. State: Replica: a81ce2cb162f4474b74f942a18579c34, State: Running, Role: LEADER
I20260811 11:12:57.525768 12800 raft_consensus.cc:2454] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a81ce2cb162f4474b74f942a18579c34 in term 1.
I20260811 11:12:57.525897 13053 consensus_queue.cc:237] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } }
I20260811 11:12:57.526813 12467 catalog_manager.cc:5495] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 reported cstate change: term changed from 0 to 1, leader changed from <none> to a81ce2cb162f4474b74f942a18579c34 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "a81ce2cb162f4474b74f942a18579c34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a81ce2cb162f4474b74f942a18579c34" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:12:57 dist-test-slave-69rg krb5kdc[11291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446776, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for HTTP/127.29.145.20@KRBTEST.COM
I20260811 11:12:57.553103 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.549452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.29.145.3:44485)
0811 11:12:57.549496 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:12:57.549498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:12:57.549509 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:12:57.549553 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:12:57.549554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:12:57.549575 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:12:57.549661 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:12:57.549876 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.550642 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.551813 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:12:57.552123 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:12:57.552186 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:12:57.552411 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:12:57.552560 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:12:57.552644 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:12:57.552985 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 11:12:57.553013 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:12:57.561426 12800 raft_consensus.cc:1261] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 1 FOLLOWER]: Refusing update from remote peer a81ce2cb162f4474b74f942a18579c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:12:57.562119 13058 consensus_queue.cc:1035] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90863b47479c4f0d8d6fd62d355e2a5f" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:12:57.564939 12648 raft_consensus.cc:1261] T 131229eca714441fbbab07e7f3811fb3 P ebec9344fe234ad380c2cdd672a06d1e [term 1 FOLLOWER]: Refusing update from remote peer a81ce2cb162f4474b74f942a18579c34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:12:57.565446 13058 consensus_queue.cc:1035] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebec9344fe234ad380c2cdd672a06d1e" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:12:57.566188 13063 mvcc.cc:205] Tried to move back new op lower bound from 7317286000862871552 to 7317286000746766336. Current Snapshot: MvccSnapshot[applied={T|T < 7317286000862871552}]
I20260811 11:12:57.569329 13062 mvcc.cc:205] Tried to move back new op lower bound from 7317286000862871552 to 7317286000746766336. Current Snapshot: MvccSnapshot[applied={T|T < 7317286000862871552 or (T in {7317286000862871552})}]
I20260811 11:12:57.569875 13068 mvcc.cc:205] Tried to move back new op lower bound from 7317286000862871552 to 7317286000746766336. Current Snapshot: MvccSnapshot[applied={T|T < 7317286000862871552}]
I20260811 11:12:57.570227 12467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:12:57.570325 12467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:12:57.571448 12467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc1cce0c0bee48a4b2076a616c02063f: Sending DeleteTablet for 3 replicas of tablet 131229eca714441fbbab07e7f3811fb3
I20260811 11:12:57.571861 12626 tablet_service.cc:1514] Processing DeleteTablet for tablet 131229eca714441fbbab07e7f3811fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:57 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:50830
I20260811 11:12:57.572005 12777 tablet_service.cc:1514] Processing DeleteTablet for tablet 131229eca714441fbbab07e7f3811fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:57 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:54198
I20260811 11:12:57.572751 12912 tablet_service.cc:1514] Processing DeleteTablet for tablet 131229eca714441fbbab07e7f3811fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:12:57 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:47940
I20260811 11:12:57.573161 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12515
I20260811 11:12:57.593969 13076 tablet_replica.cc:330] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: stopping tablet replica
I20260811 11:12:57.585567 13077 tablet_replica.cc:330] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: stopping tablet replica
I20260811 11:12:57.594365 13077 raft_consensus.cc:2229] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:12:57.594579 13076 raft_consensus.cc:2229] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:12:57.594825 13076 raft_consensus.cc:2258] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:12:57.595047 13077 raft_consensus.cc:2258] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:12:57.595736 13077 ts_tablet_manager.cc:1905] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:12:57.596101 13076 ts_tablet_manager.cc:1905] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:12:57.598140 13077 ts_tablet_manager.cc:1918] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:12:57.599833 13076 ts_tablet_manager.cc:1918] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:12:57.602871 13076 log.cc:1196] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/131229eca714441fbbab07e7f3811fb3
I20260811 11:12:57.603219 13076 ts_tablet_manager.cc:1939] T 131229eca714441fbbab07e7f3811fb3 P 90863b47479c4f0d8d6fd62d355e2a5f: Deleting consensus metadata
I20260811 11:12:57.603744 12450 catalog_manager.cc:4841] TS 90863b47479c4f0d8d6fd62d355e2a5f (127.29.145.2:43533): tablet 131229eca714441fbbab07e7f3811fb3 (table test-table [id=5dd1671065464ab78791b03d2ea65693]) successfully deleted
I20260811 11:12:57.604158 13077 log.cc:1196] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/131229eca714441fbbab07e7f3811fb3
I20260811 11:12:57.604529 13077 ts_tablet_manager.cc:1939] T 131229eca714441fbbab07e7f3811fb3 P a81ce2cb162f4474b74f942a18579c34: Deleting consensus metadata
I20260811 11:12:57.604983 12453 catalog_manager.cc:4841] TS a81ce2cb162f4474b74f942a18579c34 (127.29.145.3:44485): tablet 131229eca714441fbbab07e7f3811fb3 (table test-table [id=5dd1671065464ab78791b03d2ea65693]) successfully deleted
W20260811 11:12:57.606842 12452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:44847 (error 108)
I20260811 11:12:57.607590 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12703
I20260811 11:12:57.617898 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12858
I20260811 11:12:57.618307 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:12:57.618133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44847 (local address 127.0.0.1:50842)
0811 11:12:57.618196 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 11:12:57.618262 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 11:12:57.618445 12452 catalog_manager.cc:4584] TS ebec9344fe234ad380c2cdd672a06d1e (127.29.145.1:44847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 131229eca714441fbbab07e7f3811fb3: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44847: connect: Connection refused (error 111)
I20260811 11:12:57.628123 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 12430
I20260811 11:12:57.640298 30276 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 11:12:57.995107 30276 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 11:12:57.995258 30276 mini_ranger.cc:66] Stopping Ranger...
I20260811 11:12:58.339984 30276 mini_ranger.cc:68] Stopped Ranger
2026-08-11 11:12:58.340 UTC [11332] LOG: received smart shutdown request
2026-08-11 11:12:58.345 UTC [11332] LOG: background worker "logical replication launcher" (PID 11345) exited with exit code 1
2026-08-11 11:12:58.345 UTC [11337] LOG: shutting down
2026-08-11 11:12:58.555 UTC [11332] LOG: database system is shut down
2026-08-11T11:12:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55403 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:12:58 dist-test-slave-69rg krb5kdc[13093](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:12:58 dist-test-slave-69rg krb5kdc[13093](info): set up 2 sockets
Aug 11 11:12:58 dist-test-slave-69rg krb5kdc[13093](info): commencing operation
krb5kdc: starting...
W20260811 11:12:59.788997 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:12:59 dist-test-slave-69rg krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:12:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:12:59.922611 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:32927
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:00.008028 13109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:00.008291 13109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:00.008332 13109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:00.009465 13109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:00.009542 13109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:00.009567 13109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:00.009589 13109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:00.009617 13109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:00.011204 13109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:32927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:00.011500 13109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:00.011746 13109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:00.017307 13119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:00.017307 13117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:00.017554 13115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:00.017524 13109 server_base.cc:1034] running on GCE node
I20260811 11:13:00.017901 13109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:00.018189 13109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:00.019344 13109 hybrid_clock.cc:648] HybridClock initialized: now 1786446780019332 us; error 52 us; skew 500 ppm
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:00.022054 13109 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:00.022554 13109 webserver.cc:460] Webserver started at http://127.29.145.62:37031/ using document root <none> and password file <none>
I20260811 11:13:00.022776 13109 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:00.022816 13109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:00.022908 13109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:00.023938 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "c8e99d70926941409bc1a29234a614aa"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "13e6e5cc9daeefecff787a1acac0b14e"
server_key_iv: "bf5bffe3e225962fa15cfc8b577a0fbf"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.024272 13109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "c8e99d70926941409bc1a29234a614aa"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "13e6e5cc9daeefecff787a1acac0b14e"
server_key_iv: "bf5bffe3e225962fa15cfc8b577a0fbf"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.025584 13109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 11:13:00.026472 13127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:00.026772 13109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:00.026880 13109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "c8e99d70926941409bc1a29234a614aa"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "13e6e5cc9daeefecff787a1acac0b14e"
server_key_iv: "bf5bffe3e225962fa15cfc8b577a0fbf"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.026942 13109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:00.036844 13109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:00.039373 13109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:00.039497 13109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:00.043478 13109 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:32927
I20260811 11:13:00.043535 13185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:32927 every 8 connection(s)
I20260811 11:13:00.043905 13109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:00.044862 13186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:00.046571 13186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:13:00.048676 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13109
I20260811 11:13:00.048815 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:13:00.049016 30276 external_mini_cluster.cc:1428] Setting key 39cccfe6b784c5c6d5525030e0ea9b64
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:00.057485 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.047288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32927 (local address 127.0.0.1:49856)
0811 11:13:00.047470 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.047524 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.048011 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.048315 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.048318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.048586 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.048736 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.048740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.049879 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.049881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.050562 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.050564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.051183 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.051185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.051286 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.051844 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.051855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.053332 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.055808 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.055816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.055817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.056019 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.056346 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.056348 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.056348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.056450 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.056957 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.056964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.057033 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.057149 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.057301 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260811 11:13:00.057505 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.047458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.29.145.62:32927)
0811 11:13:00.047577 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.047581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.047619 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:13:00.048186 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.048188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.048206 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.048308 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.048887 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.049756 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.050684 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.051085 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.051180 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.053430 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.053438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.053439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.053446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.055686 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.056116 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.056119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.056120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.056245 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.056537 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.056538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.056538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.056854 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.056953 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.057224 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.057391 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":90,"thread_start_us":44,"threads_started":1}
WARNING: no policy specified for kudu/127.29.145.61@KRBTEST.COM; defaulting to no policy
I20260811 11:13:00.058517 13186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:13:00.058930 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.058804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56282)
0811 11:13:00.058861 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.058910 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 11:13:00.059106 13129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
I20260811 11:13:00.059701 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.059573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56286)
0811 11:13:00.059635 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.059669 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:13:00.059821 13186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:38499: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
I20260811 11:13:00.082362 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:38499
--webserver_interface=127.29.145.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:00.109054 13186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } attempt: 1
I20260811 11:13:00.109617 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.109463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56290)
0811 11:13:00.109543 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.109585 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 11:13:00.109994 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.109894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56304)
0811 11:13:00.109943 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.109968 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 11:13:00.110090 13186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:38499: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
W20260811 11:13:00.168452 13195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:00.168689 13195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:00.168726 13195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:00.169926 13195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:00.169992 13195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:00.170015 13195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:00.170037 13195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:00.170077 13195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:00.171594 13195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:38499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.145.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:00.171886 13195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:00.172114 13195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:00.177170 13200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:00.177217 13201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:00.177253 13195 server_base.cc:1034] running on GCE node
W20260811 11:13:00.177135 13203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:00.177551 13195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:00.177927 13195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:00.178238 13186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } attempt: 2
I20260811 11:13:00.178637 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.178502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56310)
0811 11:13:00.178565 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.178600 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 11:13:00.178943 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.178866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56324)
0811 11:13:00.178905 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.178928 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 11:13:00.179065 13186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:38499: Network error: Client connection negotiation failed: client connection to 127.29.145.61:38499: connect: Connection refused (error 111)
I20260811 11:13:00.179095 13195 hybrid_clock.cc:648] HybridClock initialized: now 1786446780179074 us; error 31 us; skew 500 ppm
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:00.181749 13195 init.cc:377] Logged in from keytab as kudu/127.29.145.61@KRBTEST.COM (short username kudu)
I20260811 11:13:00.182245 13195 webserver.cc:460] Webserver started at http://127.29.145.61:39917/ using document root <none> and password file <none>
I20260811 11:13:00.182432 13195 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:00.182472 13195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:00.182561 13195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:00.183573 13195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data/instance:
uuid: "aebaeb09cdba479d9d27344d9b4f7f91"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "2fc9c72a1d7a09f22ffbc82c2e8211a0"
server_key_iv: "2b7500b4f83364d11a1afe83ce40db20"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.184141 13195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal/instance:
uuid: "aebaeb09cdba479d9d27344d9b4f7f91"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "2fc9c72a1d7a09f22ffbc82c2e8211a0"
server_key_iv: "2b7500b4f83364d11a1afe83ce40db20"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.185665 13195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:13:00.186609 13211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:00.186882 13195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:00.186997 13195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "aebaeb09cdba479d9d27344d9b4f7f91"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "2fc9c72a1d7a09f22ffbc82c2e8211a0"
server_key_iv: "2b7500b4f83364d11a1afe83ce40db20"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.187079 13195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:00.200306 13195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:00.202289 13195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:00.202446 13195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:00.206563 13195 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.61:38499
I20260811 11:13:00.206600 13270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.61:38499 every 8 connection(s)
I20260811 11:13:00.206980 13195 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
I20260811 11:13:00.207962 13271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:00.208902 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13195
I20260811 11:13:00.209018 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-1/wal/instance
I20260811 11:13:00.209295 30276 external_mini_cluster.cc:1428] Setting key 05e3ed00375023d805d1e20604a83b8a
I20260811 11:13:00.210368 13271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.60@KRBTEST.COM; defaulting to no policy
I20260811 11:13:00.221151 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.211385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.29.145.62:32927)
0811 11:13:00.211437 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.211439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.211746 (+ 307us) server_negotiation.cc:407] Connection header received
0811 11:13:00.212203 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.212205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.212227 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.212317 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.213071 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.213681 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.215330 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.215562 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.215631 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.218453 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.218455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.218456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.218467 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.219944 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.220323 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.220324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.220325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.220378 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.220596 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.220597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.220598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.220705 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.220755 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.221011 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.221081 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:13:00.221380 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.211387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32927 (local address 127.0.0.1:49866)
0811 11:13:00.211597 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.211645 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.212100 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.212391 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.212394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.212656 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.212951 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.212957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.213839 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.213840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.215231 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.215234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.215660 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.215661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.215847 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.216444 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.216467 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.218334 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.220025 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.220033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.220034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.220254 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.220436 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.220437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.220437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.220537 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.220762 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.220772 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.220897 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.220982 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.221111 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.60@KRBTEST.COM" created.
I20260811 11:13:00.222086 13271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:13:00.229988 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.222355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56338)
0811 11:13:00.222410 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.222420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.222459 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.222748 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.222749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.222907 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.222969 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.222971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.223805 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.223806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.224483 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.224485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.224952 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.224953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.225022 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.225466 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.225470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.226942 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.228755 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.228756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.228757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.229009 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.229250 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.229251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.229251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.229323 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.229598 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.229600 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.229648 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.229722 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.229870 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:13:00.230145 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.222431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.29.145.61:38499)
0811 11:13:00.222590 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.222595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.222614 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:13:00.222643 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.222645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.222656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.222721 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.223045 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.223716 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.224571 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.224874 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.224956 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.227024 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.227032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.227032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.227038 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.228654 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.229097 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.229100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.229100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.229176 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.229390 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.229391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.229391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.229530 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.229595 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.229917 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.229982 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260811 11:13:00.230970 13271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:13:00.231357 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.231221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33034)
0811 11:13:00.231278 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.231334 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 11:13:00.231508 13215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
I20260811 11:13:00.231966 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.231843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33048)
0811 11:13:00.231899 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.231933 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 11:13:00.232079 13271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:42569: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
I20260811 11:13:00.241240 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:42569
--webserver_interface=127.29.145.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:43879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:00.270234 13186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } attempt: 3
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:13:00.278223 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.270662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56344)
0811 11:13:00.270740 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.270760 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.270833 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.270996 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.270997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.271169 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.271246 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.271248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.271952 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.271953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.272456 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.272457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.272857 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.272858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.272944 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.273473 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.273478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.275002 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.276958 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.276959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.276959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.277198 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.277431 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.277432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.277432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.277496 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.277908 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.277909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.277971 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.278053 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.278151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 11:13:00.278298 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.270761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.29.145.61:38499)
0811 11:13:00.270837 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.270839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.270850 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:13:00.270904 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.270905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.270924 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.270989 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.271335 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.271854 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.272545 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.272768 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.272846 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.275094 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.275096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.275096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.275105 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.276854 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.277281 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.277282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.277283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.277351 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.277596 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.277598 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.277598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.277792 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.277909 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.278111 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.278203 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:13:00.278636 13186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:13:00.279050 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.278924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33060)
0811 11:13:00.278975 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.279014 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 11:13:00.279418 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.279313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33070)
0811 11:13:00.279367 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.279396 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 11:13:00.279558 13186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:42569: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
I20260811 11:13:00.281277 13271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } attempt: 1
I20260811 11:13:00.281667 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.281552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33086)
0811 11:13:00.281608 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.281642 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 11:13:00.282047 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.281949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33096)
0811 11:13:00.281995 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.282022 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:13:00.282220 13271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:42569: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
I20260811 11:13:00.310711 13186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } attempt: 1
I20260811 11:13:00.311305 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.311126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33102)
0811 11:13:00.311195 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.311260 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:13:00.311705 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.311604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33104)
0811 11:13:00.311649 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.311680 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 11:13:00.311815 13186 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:42569: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
W20260811 11:13:00.331161 13281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:00.331377 13281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:00.331400 13281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:00.332465 13281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:00.332499 13281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:00.332515 13281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:00.332526 13281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:00.332538 13281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:00.334066 13281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:43879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:32927,127.29.145.61:38499,127.29.145.60:42569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:42569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.145.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:00.334275 13281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:00.334448 13281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:13:00.339532 13281 server_base.cc:1034] running on GCE node
W20260811 11:13:00.339486 13289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:00.339731 13291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:00.339486 13288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:00.340137 13281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:00.340540 13281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:00.341732 13281 hybrid_clock.cc:648] HybridClock initialized: now 1786446780341718 us; error 49 us; skew 500 ppm
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:00.344515 13281 init.cc:377] Logged in from keytab as kudu/127.29.145.60@KRBTEST.COM (short username kudu)
I20260811 11:13:00.344973 13281 webserver.cc:460] Webserver started at http://127.29.145.60:33343/ using document root <none> and password file <none>
I20260811 11:13:00.345238 13281 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:00.345319 13281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:00.345474 13281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:00.346632 13281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data/instance:
uuid: "645d9596cf3c4c68b5bde10e82cceec2"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "182d335ef4d7003674f418bb616f8a14"
server_key_iv: "8edc7188480ceabc5b09972cdcd93387"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.347007 13281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal/instance:
uuid: "645d9596cf3c4c68b5bde10e82cceec2"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "182d335ef4d7003674f418bb616f8a14"
server_key_iv: "8edc7188480ceabc5b09972cdcd93387"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.348441 13281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:00.349229 13298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:00.349443 13281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:00.349566 13281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "645d9596cf3c4c68b5bde10e82cceec2"
format_stamp: "Formatted at 2026-08-11 11:13:00 on dist-test-slave-69rg"
server_key: "182d335ef4d7003674f418bb616f8a14"
server_key_iv: "8edc7188480ceabc5b09972cdcd93387"
server_key_version: "encryptionkey@0"
I20260811 11:13:00.349647 13281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:00.350404 13271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } attempt: 2
I20260811 11:13:00.350839 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.350709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33116)
0811 11:13:00.350774 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.350810 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 11:13:00.351214 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.351089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33126)
0811 11:13:00.351163 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.351189 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 11:13:00.351301 13271 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:42569: Network error: Client connection negotiation failed: client connection to 127.29.145.60:42569: connect: Connection refused (error 111)
I20260811 11:13:00.368940 13281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:00.371493 13281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:00.371667 13281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:00.375795 13281 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.60:42569
I20260811 11:13:00.375842 13358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.60:42569 every 8 connection(s)
I20260811 11:13:00.376238 13281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
I20260811 11:13:00.377152 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:00.378304 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13281
I20260811 11:13:00.378413 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786446611237378-30276-0/minicluster-data/master-2/wal/instance
I20260811 11:13:00.378600 30276 external_mini_cluster.cc:1428] Setting key 32071974defd2a1c5ede32914b45a03e
I20260811 11:13:00.379141 13359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:13:00.386966 13186 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } attempt: 2
I20260811 11:13:00.388772 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.379938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49880 (local address 127.29.145.62:32927)
0811 11:13:00.379990 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.379992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.380098 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:13:00.380168 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.380169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.380181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.380282 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.380854 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.381361 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.382368 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.382630 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.382709 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.385083 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.385085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.385086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.385093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.386797 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.387632 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.387634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.387634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.387699 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.388016 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.388017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.388018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.388162 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.388238 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.388561 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.388655 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 11:13:00.389273 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.379849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32927 (local address 127.0.0.1:49880)
0811 11:13:00.380004 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.380042 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.380113 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.380315 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.380318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.380588 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.380753 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.380757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.381450 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.381451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.382262 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.382265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.382722 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.382723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.382832 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.383376 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.383386 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.384985 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.386902 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.386910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.386912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.387535 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.387779 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.387780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.387781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.387923 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.388286 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.388293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.388416 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.388517 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.388822 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260811 11:13:00.389652 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.379163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.29.145.60:42569)
0811 11:13:00.379388 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.379398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.379433 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:13:00.379919 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.379928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.379993 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.380153 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.380801 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.381584 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.382405 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.382755 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.382837 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.385536 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.385545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.385545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.385554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.387620 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.388695 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.388699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.388699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.388762 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.389005 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.389006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.389007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.389176 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.389395 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.389443 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.389529 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":104,"threads_started":1}
I20260811 11:13:00.390105 13359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:13:00.396979 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.387253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33136)
0811 11:13:00.387315 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.387327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.387378 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.387665 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.387666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.387835 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.387879 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.387881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.388824 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.388828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.389769 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.389773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.390219 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.390221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.390279 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.390733 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.390738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.392070 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.395780 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.395782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.395782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.396005 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.396253 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.396255 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.396255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.396321 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.396610 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.396612 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.396664 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.396744 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.396869 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 11:13:00.397449 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.387351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.29.145.60:42569)
0811 11:13:00.387511 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.387513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.387522 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:00.387542 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.387542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.387553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.387653 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.387972 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.388594 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.389920 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.390134 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.390219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.392162 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.392164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.392164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.392171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.394563 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.396099 (+ 1536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.396100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.396100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.396165 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.396395 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.396396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.396396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.396527 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.396597 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.397207 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.397296 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260811 11:13:00.399322 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.390423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:38499 (local address 127.0.0.1:56358)
0811 11:13:00.390458 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.390470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.390509 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.391257 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.391258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.391414 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.391467 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.391469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.392345 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.392346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.393177 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.393181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.393662 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.393663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.393742 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.394709 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.394714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.396020 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.398220 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.398222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.398393 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.398616 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.398618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.398677 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.398992 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.398994 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.399037 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.399123 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.399229 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 11:13:00.399326 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.390541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.29.145.61:38499)
0811 11:13:00.390591 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.390593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.390600 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:13:00.390624 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.390625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.390634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.390749 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.391546 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.392263 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.393284 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.393582 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.393645 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.396115 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.396116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.396117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.396125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.398116 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398482 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.398483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.398483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.398550 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398760 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.398761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.398762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.398920 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.399000 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.399163 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.399241 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:13:00.399672 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.390792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.29.145.62:32927)
0811 11:13:00.390830 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.390832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.390854 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:13:00.390927 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.390928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.390937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.390992 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.391475 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.392164 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.393019 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.393210 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.393290 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.395702 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.395703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.395704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.395711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.398137 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398522 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.398524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.398525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.398579 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.398791 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.398792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.398792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.398928 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.399128 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.399516 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.399591 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:13:00.399683 13186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: Bootstrap starting.
I20260811 11:13:00.400158 13359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:13:00.401369 13186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:00.401716 13186 log.cc:824] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:00.402661 13186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: No bootstrap required, opened a new log
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:13:00.405015 13186 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.405185 13186 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:00.405225 13186 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e99d70926941409bc1a29234a614aa, State: Initialized, Role: FOLLOWER
I20260811 11:13:00.405458 13186 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.405786 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.405885 13186 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:00.405936 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.408488 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.400459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33140)
0811 11:13:00.400532 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.400542 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.400582 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.400807 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.400808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.401127 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.401173 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.401177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.402110 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.402111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.402825 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.402828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.403317 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.403318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.403376 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.403707 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.403712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.405355 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.407364 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.407365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.407365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.407547 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.407818 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.407818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.407819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.407883 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.408159 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.408160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.408200 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.408276 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.408378 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 11:13:00.408588 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.400492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.29.145.60:42569)
0811 11:13:00.400569 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.400572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.400593 (+ 21us) server_negotiation.cc:407] Connection header received
0811 11:13:00.400646 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.400648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.400666 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.400748 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.401288 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.402022 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.402942 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.403229 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.403298 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.405425 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.405426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.405427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.405434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.407268 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.407645 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.407646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.407647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.407700 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.407961 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.407962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.407963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.408094 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.408169 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.408300 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.408435 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":33}
I20260811 11:13:00.409159 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.400735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.29.145.61:38499)
0811 11:13:00.400770 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.400772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.400780 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:13:00.400821 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.400823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.400832 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.400880 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.401359 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.401811 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.402636 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.402849 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.402947 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.405551 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.405553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.405553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.405560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.407622 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.408102 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.408103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.408103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.408168 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.408435 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.408436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.408436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.408584 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.408790 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.408989 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.409072 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 11:13:00.410374 13377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:13:00.410485 13377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:13:00.410480 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:13:00.410494 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: Bootstrap starting.
I20260811 11:13:00.411134 13359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:00.411355 13359 log.cc:824] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:00.412128 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: No bootstrap required, opened a new log
I20260811 11:13:00.409848 13109 master_runner.cc:186] Error getting master registration for 127.29.145.61:38499: OK, instance_id { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" instance_seqno: 1786446780205355 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:13:00.413077 13109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:13:00.414382 13359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.414522 13359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:00.414556 13359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645d9596cf3c4c68b5bde10e82cceec2, State: Initialized, Role: FOLLOWER
I20260811 11:13:00.414675 13359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.414950 13389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.415078 13389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.414963 13359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 11:13:00.417795 13402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:13:00.417937 13402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:13:00.422648 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.412188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.29.145.62:32927)
0811 11:13:00.412238 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.412240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.412510 (+ 270us) server_negotiation.cc:407] Connection header received
0811 11:13:00.412602 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.412603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.412613 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.412642 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.413447 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.414199 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.415223 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.415532 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.415580 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.418681 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.418683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.418683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.418691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.420403 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.420781 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.420783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.420783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.421097 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.421316 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.421317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.421318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.421445 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.421549 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.421974 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.422529 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:13:00.420224 13281 master_runner.cc:186] Error getting master registration for 127.29.145.61:38499: OK, instance_id { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" instance_seqno: 1786446780205355 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:13:00.422812 13281 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:13:00.425055 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.412120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.29.145.61:38499)
0811 11:13:00.412156 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.412158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.417469 (+ 5311us) server_negotiation.cc:407] Connection header received
0811 11:13:00.417567 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.417568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.417580 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.417611 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.418315 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.419127 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.420171 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.420451 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.420500 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.422044 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.422045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.422046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.422053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.423658 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.424077 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.424078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.424079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.424145 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.424382 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.424383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.424383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.424528 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.424636 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.424862 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.424950 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:13:00.426815 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.417277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.29.145.60:42569)
0811 11:13:00.417314 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.417315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.418776 (+ 1461us) server_negotiation.cc:407] Connection header received
0811 11:13:00.418870 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.418871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.418881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.418912 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.419590 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.420325 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.421337 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.421614 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.421657 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.423524 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.423525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.423526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.423533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.425433 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.425896 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.425897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.425897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.425962 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.426225 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.426226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.426227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.426367 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.426488 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.426638 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.426719 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:13:00.442440 13271 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } attempt: 3
Aug 11 11:13:00 dist-test-slave-69rg krb5kdc[13093](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446780, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:13:00.450806 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.442747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:42569 (local address 127.0.0.1:33162)
0811 11:13:00.442808 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:00.442819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:00.442860 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:00.443003 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:00.443005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:00.443200 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:00.443254 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.443256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.444175 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.444177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.444997 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:00.445000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.445491 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.445492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:00.445556 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.446097 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:00.446102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:00.447657 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:00.449598 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.449599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.449600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.449853 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.450059 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:00.450060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:00.450060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:00.450128 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:00.450453 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:00.450455 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:00.450519 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 11:13:00.450590 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:13:00.450687 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 11:13:00.451632 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:00.442793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.29.145.60:42569)
0811 11:13:00.442836 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:00.442838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:00.442846 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:13:00.442912 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:00.442914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:00.442923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:00.442986 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:00.443336 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.444071 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.445093 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:00.445406 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:00.445478 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:00.447752 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:00.447753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:00.447754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:00.447768 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:00.449511 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.449919 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.449920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.449920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.449982 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:00.450199 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:00.450200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:00.450200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:00.450335 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:00.450411 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:00.451449 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0811 11:13:00.451538 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:13:00.453274 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: Bootstrap starting.
I20260811 11:13:00.454062 13271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:00.454394 13271 log.cc:824] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:00.455329 13271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: No bootstrap required, opened a new log
I20260811 11:13:00.457341 13271 raft_consensus.cc:348] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.457486 13271 raft_consensus.cc:374] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:00.457521 13271 raft_consensus.cc:729] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aebaeb09cdba479d9d27344d9b4f7f91, State: Initialized, Role: FOLLOWER
I20260811 11:13:00.457665 13271 consensus_queue.cc:260] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.458016 13271 sys_catalog.cc:564] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:00.457991 13407 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.458204 13407 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:13:00.459956 13419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:13:00.460062 13419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:13:00.461752 13195 master_runner.cc:263] No leader master found from master aebaeb09cdba479d9d27344d9b4f7f91
I20260811 11:13:00.461809 13195 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:13:00.473306 13366 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:00.473433 13366 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.473685 13366 leader_election.cc:288] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aebaeb09cdba479d9d27344d9b4f7f91 (127.29.145.61:38499), 645d9596cf3c4c68b5bde10e82cceec2 (127.29.145.60:42569)
I20260811 11:13:00.474022 13330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8e99d70926941409bc1a29234a614aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645d9596cf3c4c68b5bde10e82cceec2" is_pre_election: true
I20260811 11:13:00.474045 13243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8e99d70926941409bc1a29234a614aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" is_pre_election: true
I20260811 11:13:00.474181 13330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8e99d70926941409bc1a29234a614aa in term 0.
I20260811 11:13:00.474180 13243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8e99d70926941409bc1a29234a614aa in term 0.
I20260811 11:13:00.474359 13129 leader_election.cc:302] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aebaeb09cdba479d9d27344d9b4f7f91, c8e99d70926941409bc1a29234a614aa; no voters:
I20260811 11:13:00.474483 13366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:13:00.474525 13366 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:13:00.474547 13366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:00.475538 13366 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.475672 13366 leader_election.cc:288] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [CANDIDATE]: Term 1 election: Requested vote from peers aebaeb09cdba479d9d27344d9b4f7f91 (127.29.145.61:38499), 645d9596cf3c4c68b5bde10e82cceec2 (127.29.145.60:42569)
I20260811 11:13:00.475850 13330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8e99d70926941409bc1a29234a614aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645d9596cf3c4c68b5bde10e82cceec2"
I20260811 11:13:00.475929 13330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:00.476761 13330 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8e99d70926941409bc1a29234a614aa in term 1.
I20260811 11:13:00.476943 13130 leader_election.cc:302] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645d9596cf3c4c68b5bde10e82cceec2, c8e99d70926941409bc1a29234a614aa; no voters:
I20260811 11:13:00.477130 13243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8e99d70926941409bc1a29234a614aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aebaeb09cdba479d9d27344d9b4f7f91"
I20260811 11:13:00.477216 13243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:00.478120 13243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8e99d70926941409bc1a29234a614aa in term 1.
I20260811 11:13:00.478360 13366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:00.478535 13366 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [term 1 LEADER]: Becoming Leader. State: Replica: c8e99d70926941409bc1a29234a614aa, State: Running, Role: LEADER
I20260811 11:13:00.478631 13366 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } }
I20260811 11:13:00.479008 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8e99d70926941409bc1a29234a614aa. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.479090 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:00.479316 13423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:00.479507 13423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:00.481424 13330 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [term 1 FOLLOWER]: Refusing update from remote peer c8e99d70926941409bc1a29234a614aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:00.481719 13366 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:00.483848 13389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8e99d70926941409bc1a29234a614aa. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.483937 13389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.484261 13243 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [term 1 FOLLOWER]: Refusing update from remote peer c8e99d70926941409bc1a29234a614aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:00.484486 13366 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:00.485630 13407 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8e99d70926941409bc1a29234a614aa. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.485721 13407 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.487257 13421 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.487349 13421 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:00.487793 13407 sys_catalog.cc:455] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.487879 13407 sys_catalog.cc:458] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.488068 13427 mvcc.cc:205] Tried to move back new op lower bound from 7317286012849508352 to 7317286012840996864. Current Snapshot: MvccSnapshot[applied={T|T < 7317286012849508352}]
I20260811 11:13:00.488250 13431 mvcc.cc:205] Tried to move back new op lower bound from 7317286012849508352 to 7317286012840996864. Current Snapshot: MvccSnapshot[applied={T|T < 7317286012849508352}]
I20260811 11:13:00.488541 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.488626 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:00.488925 13389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c8e99d70926941409bc1a29234a614aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8e99d70926941409bc1a29234a614aa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32927 } } peers { permanent_uuid: "aebaeb09cdba479d9d27344d9b4f7f91" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 38499 } } peers { permanent_uuid: "645d9596cf3c4c68b5bde10e82cceec2" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 42569 } } }
I20260811 11:13:00.489013 13389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:13:00.489692 13428 mvcc.cc:205] Tried to move back new op lower bound from 7317286012849508352 to 7317286012840996864. Current Snapshot: MvccSnapshot[applied={T|T < 7317286012849508352}]
I20260811 11:13:00.490274 13423 catalog_manager.cc:1347] Generated new cluster ID: c619fbddbad9443fa41a75c7e91e4fc0
I20260811 11:13:00.490326 13423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:13:00.517711 13423 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:13:00.518522 13423 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:00.530406 13423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8e99d70926941409bc1a29234a614aa: Generated new TSK 0
I20260811 11:13:00.530661 13423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:13:01.418604 13402 catalog_manager.cc:1259] Loaded cluster ID: c619fbddbad9443fa41a75c7e91e4fc0
I20260811 11:13:01.418677 13402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: loading cluster ID for follower catalog manager: success
I20260811 11:13:01.419626 13402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: acquiring CA information for follower catalog manager: success
I20260811 11:13:01.419945 13402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 645d9596cf3c4c68b5bde10e82cceec2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:13:01.460808 13419 catalog_manager.cc:1259] Loaded cluster ID: c619fbddbad9443fa41a75c7e91e4fc0
I20260811 11:13:01.460928 13419 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: loading cluster ID for follower catalog manager: success
I20260811 11:13:01.462038 13419 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: acquiring CA information for follower catalog manager: success
I20260811 11:13:01.462467 13419 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aebaeb09cdba479d9d27344d9b4f7f91: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:13:01.588853 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13109
I20260811 11:13:01.599642 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13195
I20260811 11:13:01.607400 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13281
2026-08-11T11:13:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2938 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:13:01 dist-test-slave-69rg krb5kdc[13446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:13:01 dist-test-slave-69rg krb5kdc[13446](info): set up 2 sockets
Aug 11 11:13:01 dist-test-slave-69rg krb5kdc[13446](info): commencing operation
krb5kdc: starting...
W20260811 11:13:02.775090 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:13:02 dist-test-slave-69rg krb5kdc[13446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:13:02.908528 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34875
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:46799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:34875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:03.001931 13462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:03.002153 13462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:03.002182 13462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:03.003446 13462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:03.003495 13462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:03.003511 13462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:03.003526 13462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:03.003587 13462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:03.005283 13462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:46799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:34875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:34875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:03.005581 13462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:03.005883 13462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:03.011312 13468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:03.011335 13471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:03.011335 13469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:03.011422 13462 server_base.cc:1034] running on GCE node
I20260811 11:13:03.011864 13462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:03.012117 13462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:03.013269 13462 hybrid_clock.cc:648] HybridClock initialized: now 1786446783013250 us; error 53 us; skew 500 ppm
Aug 11 11:13:03 dist-test-slave-69rg krb5kdc[13446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446783, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:03.015918 13462 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:03.016316 13462 webserver.cc:460] Webserver started at http://127.29.145.62:36097/ using document root <none> and password file <none>
I20260811 11:13:03.016498 13462 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:03.016538 13462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:03.016628 13462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:03.017663 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "9ab0a79410bc44759b2c7b91b10a5151"
format_stamp: "Formatted at 2026-08-11 11:13:03 on dist-test-slave-69rg"
server_key: "a8e4ba6cfcb37920320ebaf47defb8a4"
server_key_iv: "22c01f36ad89645f9149cbe24981e6bf"
server_key_version: "encryptionkey@0"
I20260811 11:13:03.018064 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "9ab0a79410bc44759b2c7b91b10a5151"
format_stamp: "Formatted at 2026-08-11 11:13:03 on dist-test-slave-69rg"
server_key: "a8e4ba6cfcb37920320ebaf47defb8a4"
server_key_iv: "22c01f36ad89645f9149cbe24981e6bf"
server_key_version: "encryptionkey@0"
I20260811 11:13:03.019476 13462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:03.020277 13479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:03.020501 13462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:03.020598 13462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "9ab0a79410bc44759b2c7b91b10a5151"
format_stamp: "Formatted at 2026-08-11 11:13:03 on dist-test-slave-69rg"
server_key: "a8e4ba6cfcb37920320ebaf47defb8a4"
server_key_iv: "22c01f36ad89645f9149cbe24981e6bf"
server_key_version: "encryptionkey@0"
I20260811 11:13:03.020655 13462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:03.033608 13462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:03.036016 13462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:03.036135 13462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:03.040489 13462 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:34875
I20260811 11:13:03.040767 13534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:34875 every 8 connection(s)
I20260811 11:13:03.041007 13462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:03.042091 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:03.044486 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151: Bootstrap starting.
I20260811 11:13:03.044510 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13462
I20260811 11:13:03.044642 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:13:03.044863 30276 external_mini_cluster.cc:1428] Setting key 82ce9046d699530a182490de57c5928e
I20260811 11:13:03.045222 13535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:03.045585 13535 log.cc:824] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:03.046398 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151: No bootstrap required, opened a new log
I20260811 11:13:03.048794 13535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } }
I20260811 11:13:03.048976 13535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:03.049009 13535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab0a79410bc44759b2c7b91b10a5151, State: Initialized, Role: FOLLOWER
I20260811 11:13:03.049129 13535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } }
I20260811 11:13:03.049202 13535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:03.049233 13535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:03.049280 13535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:03.049957 13535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } }
I20260811 11:13:03.050081 13535 leader_election.cc:302] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ab0a79410bc44759b2c7b91b10a5151; no voters:
I20260811 11:13:03.050269 13535 leader_election.cc:288] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:13:03.050467 13541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:03.050567 13535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:03.050673 13541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [term 1 LEADER]: Becoming Leader. State: Replica: 9ab0a79410bc44759b2c7b91b10a5151, State: Running, Role: LEADER
I20260811 11:13:03.050817 13541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } }
I20260811 11:13:03.051182 13542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ab0a79410bc44759b2c7b91b10a5151. Latest consensus state: current_term: 1 leader_uuid: "9ab0a79410bc44759b2c7b91b10a5151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } } }
I20260811 11:13:03.051275 13542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:03.052198 13548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 11:13:03 dist-test-slave-69rg krb5kdc[13446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:03.052727 13548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:03.052759 13540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ab0a79410bc44759b2c7b91b10a5151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ab0a79410bc44759b2c7b91b10a5151" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 34875 } } }
I20260811 11:13:03.052846 13540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151 [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:03.054478 13548 catalog_manager.cc:1347] Generated new cluster ID: f6c42132d207416aa5501fe1b8b6d072
I20260811 11:13:03.054543 13548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:13:03.061401 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:03.045415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.29.145.62:34875)
0811 11:13:03.045635 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:03.045645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:03.045683 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:13:03.046305 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:03.046313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:03.046364 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:03.046532 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:03.047371 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:03.048296 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:03.049393 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:03.049876 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:03.049973 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:03.053251 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:03.053256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:03.053256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:03.053276 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:03.059415 (+ 6139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:03.059881 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:03.059886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:03.059886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:03.060025 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:03.060270 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:03.060272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:03.060272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:03.060604 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:03.060774 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:03.060866 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 11:13:03.061152 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260811 11:13:03.062480 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:13:03.072194 13548 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:13:03.072753 13548 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:03.078202 13548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ab0a79410bc44759b2c7b91b10a5151: Generated new TSK 0
I20260811 11:13:03.078414 13548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:13:03.116825 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13462
2026-08-11T11:13:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1510 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:13:03 dist-test-slave-69rg krb5kdc[13565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:13:03 dist-test-slave-69rg krb5kdc[13565](info): set up 2 sockets
Aug 11 11:13:03 dist-test-slave-69rg krb5kdc[13565](info): commencing operation
krb5kdc: starting...
W20260811 11:13:04.291574 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:13:04.422875 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:38291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:04.506055 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:04.506296 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:04.506332 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:04.507458 13580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:04.507534 13580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:04.507560 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:04.507587 13580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:04.507617 13580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:04.509138 13580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:38291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:04.509413 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:04.509649 13580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:04.514873 13588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:04.514988 13580 server_base.cc:1034] running on GCE node
W20260811 11:13:04.514874 13585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:04.514916 13586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:04.515389 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:04.515656 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:04.516850 13580 hybrid_clock.cc:648] HybridClock initialized: now 1786446784516824 us; error 37 us; skew 500 ppm
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:04.519558 13580 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:04.519999 13580 webserver.cc:460] Webserver started at http://127.29.145.62:38669/ using document root <none> and password file <none>
I20260811 11:13:04.520277 13580 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:04.520349 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:04.520498 13580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:04.521557 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "23b2a23d923ceae23f403df39bbe0feb"
server_key_iv: "3eb904c861648c05715c9be1be9374bc"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.521986 13580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "23b2a23d923ceae23f403df39bbe0feb"
server_key_iv: "3eb904c861648c05715c9be1be9374bc"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.523411 13580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:04.524422 13596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.524593 13580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:04.524751 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "23b2a23d923ceae23f403df39bbe0feb"
server_key_iv: "3eb904c861648c05715c9be1be9374bc"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.524835 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:04.548291 13580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:04.550348 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:04.550506 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:04.554877 13580 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:38291
I20260811 11:13:04.554939 13656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:38291 every 8 connection(s)
I20260811 11:13:04.555322 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:04.556378 13657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:04.558583 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13580
I20260811 11:13:04.558676 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:13:04.558794 13657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap starting.
I20260811 11:13:04.558910 30276 external_mini_cluster.cc:1428] Setting key 09988817b816c0c8156a17d9b19425c1
I20260811 11:13:04.559581 13657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:04.559957 13657 log.cc:824] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:04.560811 13657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: No bootstrap required, opened a new log
I20260811 11:13:04.563108 13657 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:04.563279 13657 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:04.563313 13657 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Initialized, Role: FOLLOWER
I20260811 11:13:04.563443 13657 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:04.563549 13657 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:04.563582 13657 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:04.563627 13657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:04.564359 13657 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:04.564496 13657 leader_election.cc:302] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc1758b60c2c4638b30b3ad6dbef0bfd; no voters:
I20260811 11:13:04.564684 13657 leader_election.cc:288] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:13:04.564741 13664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:04.564958 13664 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 LEADER]: Becoming Leader. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Running, Role: LEADER
I20260811 11:13:04.565023 13657 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:04.565177 13664 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:04.565954 13666 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc1758b60c2c4638b30b3ad6dbef0bfd. Latest consensus state: current_term: 1 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:04.566052 13666 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:04.566130 13665 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:04.566206 13665 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
W20260811 11:13:04.566991 13679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:13:04.567046 13679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:13:04.567098 13673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:04.567281 13673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:04.568903 13673 catalog_manager.cc:1347] Generated new cluster ID: 59d97f1bee204c199baafcfcb4a6c49f
I20260811 11:13:04.568958 13673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:13:04.571178 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:04.559518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.29.145.62:38291)
0811 11:13:04.559710 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:04.559718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:04.559744 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:13:04.560323 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:04.560333 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:04.560397 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:04.560551 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:04.561368 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.562329 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.563469 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.564088 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.564214 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:04.567444 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:04.567449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:04.567449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:04.567461 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:04.569591 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.570049 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.570052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.570053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.570163 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.570353 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.570354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.570354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.570639 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:04.570766 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:04.570847 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 11:13:04.570973 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260811 11:13:04.575608 13673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:13:04.576133 13673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:04.581400 13673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Generated new TSK 0
I20260811 11:13:04.581683 13673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:13:04.613720 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:04.699091 13685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:04.699339 13685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:04.699378 13685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:04.700510 13685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:04.700553 13685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:04.700582 13685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:13:04.702114 13685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:04.702353 13685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:04.702638 13685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:04.708212 13690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:04.708202 13691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:04.708204 13693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:04.708401 13685 server_base.cc:1034] running on GCE node
I20260811 11:13:04.708676 13685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:04.708886 13685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:04.710024 13685 hybrid_clock.cc:648] HybridClock initialized: now 1786446784710007 us; error 30 us; skew 500 ppm
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:04.712702 13685 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:13:04.713174 13685 webserver.cc:460] Webserver started at http://127.29.145.1:40259/ using document root <none> and password file <none>
I20260811 11:13:04.713361 13685 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:04.713400 13685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:04.713490 13685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:04.714569 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "1026bf93f7274f919c9241a61f8c5fcf"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "010adc3f6399e0ad4eb74690f2f26c57"
server_key_iv: "fb5a2146baf69aed2eb522a5fadb1702"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.714892 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "1026bf93f7274f919c9241a61f8c5fcf"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "010adc3f6399e0ad4eb74690f2f26c57"
server_key_iv: "fb5a2146baf69aed2eb522a5fadb1702"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.716292 13685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:13:04.717058 13702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.717252 13685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:04.717330 13685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "1026bf93f7274f919c9241a61f8c5fcf"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "010adc3f6399e0ad4eb74690f2f26c57"
server_key_iv: "fb5a2146baf69aed2eb522a5fadb1702"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.717386 13685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:04.731942 13685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:04.734321 13685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:04.734438 13685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:04.734725 13685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:04.735082 13685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:04.735126 13685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.735172 13685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:04.735189 13685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.741214 13685 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:35081
I20260811 11:13:04.741281 13829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:35081 every 8 connection(s)
I20260811 11:13:04.741598 13685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:04.750226 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13685
I20260811 11:13:04.750349 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:13:04.750569 30276 external_mini_cluster.cc:1428] Setting key 2b20f61549b3ca87649d6cbad8d8467d
I20260811 11:13:04.751943 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:04.742378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:39653 (local address 127.29.145.62:38291)
0811 11:13:04.742457 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:04.742459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:04.742632 (+ 173us) server_negotiation.cc:407] Connection header received
0811 11:13:04.743128 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:04.743129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:04.743141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:04.743205 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:04.744158 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.744718 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.745848 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.746067 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.746133 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:04.748589 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:04.748591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:04.748592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:04.748600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:04.750239 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.750783 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.750784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.750784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.750838 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.751167 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.751168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.751168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.751332 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:04.751421 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:04.751708 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 11:13:04.751810 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:13:04.751963 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:04.742322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:39653)
0811 11:13:04.742493 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:04.742538 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:04.743030 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:04.743259 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:04.743263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:04.743840 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:04.744036 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:04.744040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.744841 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.744842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:04.745701 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:04.745704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.746178 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.746179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:04.746398 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:04.747023 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:04.747032 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:04.748469 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:04.750348 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:04.750358 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:04.750361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:04.750695 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:04.750934 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:04.750934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:04.750935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:04.751095 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:04.751414 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:04.751422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:04.751567 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:13:04.751663 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:13:04.751792 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260811 11:13:04.753664 13830 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:04.753796 13830 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:04.754009 13830 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:04.754634 13615 ts_manager.cc:194] Registered new tserver with Master: 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081)
I20260811 11:13:04.755548 13615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:39653
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:13:04.786202 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:04.874055 13836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:04.874372 13836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:04.874435 13836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:04.875576 13836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:04.875615 13836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:04.875641 13836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:13:04.877036 13836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:04.877226 13836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:04.877424 13836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:04.882565 13843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:04.882613 13842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:04.882607 13846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:04.882899 13836 server_base.cc:1034] running on GCE node
I20260811 11:13:04.883101 13836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:04.883301 13836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:04.884456 13836 hybrid_clock.cc:648] HybridClock initialized: now 1786446784884441 us; error 30 us; skew 500 ppm
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:04.887187 13836 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:13:04.887742 13836 webserver.cc:460] Webserver started at http://127.29.145.2:41329/ using document root <none> and password file <none>
I20260811 11:13:04.887984 13836 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:04.888031 13836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:04.888167 13836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:04.889339 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "5812faeebc9b4edeb5450b55cfe93334"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "dcd3eeba15b49d689e35cfe504f737d4"
server_key_iv: "a5e28fafdb1fe936d4bfe842012083ff"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.889725 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "5812faeebc9b4edeb5450b55cfe93334"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "dcd3eeba15b49d689e35cfe504f737d4"
server_key_iv: "a5e28fafdb1fe936d4bfe842012083ff"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.891196 13836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:04.891947 13853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.892170 13836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:04.892268 13836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "5812faeebc9b4edeb5450b55cfe93334"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "dcd3eeba15b49d689e35cfe504f737d4"
server_key_iv: "a5e28fafdb1fe936d4bfe842012083ff"
server_key_version: "encryptionkey@0"
I20260811 11:13:04.892343 13836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:04.903522 13836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:04.906121 13836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:04.906272 13836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:04.906534 13836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:04.906878 13836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:04.906934 13836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.906978 13836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:04.907006 13836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:04.913295 13836 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35623
I20260811 11:13:04.913341 13985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35623 every 8 connection(s)
I20260811 11:13:04.913681 13836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:13:04 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446784, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:04.923224 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13836
I20260811 11:13:04.923336 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:13:04.923560 30276 external_mini_cluster.cc:1428] Setting key f6f9c4903f9eb742b41fe5cf2edd1dfe
I20260811 11:13:04.923800 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:04.914478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:56777 (local address 127.29.145.62:38291)
0811 11:13:04.914536 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:04.914538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:04.914915 (+ 377us) server_negotiation.cc:407] Connection header received
0811 11:13:04.915469 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:04.915470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:04.915480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:04.915539 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:04.916461 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.916952 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.918061 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.918261 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.918319 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:04.920455 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:04.920457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:04.920457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:04.920464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:04.922097 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.922634 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.922635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.922635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.922708 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:04.922987 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:04.922988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:04.922988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:04.923196 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:04.923277 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:04.923599 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 11:13:04.923720 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 11:13:04.924039 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:04.914471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:56777)
0811 11:13:04.914813 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:04.914850 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:04.915372 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:04.915590 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:04.915593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:04.916184 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:04.916380 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:04.916384 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.917061 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.917062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:04.917982 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:04.917985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:04.918342 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:04.918343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:04.918453 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:04.918912 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:04.918920 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:04.920381 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:04.922317 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:04.922324 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:04.922326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:04.922556 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:04.922792 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:04.922793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:04.922793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:04.922895 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:04.923280 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:04.923317 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:04.923463 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 11:13:04.923552 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:13:04.923766 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":261,"threads_started":1}
I20260811 11:13:04.924497 13986 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:04.924597 13986 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:04.924798 13986 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:04.925177 13615 ts_manager.cc:194] Registered new tserver with Master: 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
I20260811 11:13:04.925567 13615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:56777
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:13:04.953291 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:05.045708 13995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:05.046023 13995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:05.046054 13995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:05.047360 13995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:05.047410 13995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:05.047441 13995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:13:05.049085 13995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:05.049364 13995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:05.049644 13995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:05.055387 14002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.055361 14004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.055512 13995 server_base.cc:1034] running on GCE node
W20260811 11:13:05.055372 14000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.055817 13995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:05.056093 13995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:05.057276 13995 hybrid_clock.cc:648] HybridClock initialized: now 1786446785057261 us; error 49 us; skew 500 ppm
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:05.060261 13995 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:13:05.060822 13995 webserver.cc:460] Webserver started at http://127.29.145.3:43703/ using document root <none> and password file <none>
I20260811 11:13:05.061101 13995 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:05.061183 13995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:05.061347 13995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:05.062784 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "d5a11b2c04c643eabb36394e50688fb6"
format_stamp: "Formatted at 2026-08-11 11:13:05 on dist-test-slave-69rg"
server_key: "923d75486745acb4c711560009c2762a"
server_key_iv: "924ca060145c9981154d97bd9e2613c4"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.063196 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "d5a11b2c04c643eabb36394e50688fb6"
format_stamp: "Formatted at 2026-08-11 11:13:05 on dist-test-slave-69rg"
server_key: "923d75486745acb4c711560009c2762a"
server_key_iv: "924ca060145c9981154d97bd9e2613c4"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.064944 13995 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:13:05.065961 14011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.066205 13995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:05.066313 13995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "d5a11b2c04c643eabb36394e50688fb6"
format_stamp: "Formatted at 2026-08-11 11:13:05 on dist-test-slave-69rg"
server_key: "923d75486745acb4c711560009c2762a"
server_key_iv: "924ca060145c9981154d97bd9e2613c4"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.066414 13995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:05.091921 13995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:05.094316 13995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:05.094503 13995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:05.094789 13995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:05.095206 13995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:05.095265 13995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.095305 13995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:05.095336 13995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.102084 13995 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:45049
I20260811 11:13:05.102131 14135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:45049 every 8 connection(s)
I20260811 11:13:05.102502 13995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:05.109793 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 13995
I20260811 11:13:05.109952 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:13:05.110184 30276 external_mini_cluster.cc:1428] Setting key b8175f624d6f869eed3b7c2a23e85c00
I20260811 11:13:05.113297 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.103371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:46969 (local address 127.29.145.62:38291)
0811 11:13:05.103443 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.103446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.103719 (+ 273us) server_negotiation.cc:407] Connection header received
0811 11:13:05.104314 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.104315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.104330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.104401 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:05.105471 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.106089 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.107192 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.107406 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.107494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.109975 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:05.109977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:05.109977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:05.109988 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:05.111864 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.112306 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.112307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.112308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.112373 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.112643 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.112645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.112645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.112773 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:05.112843 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.113147 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.113231 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:13:05.113481 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.103306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:46969)
0811 11:13:05.103547 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.103587 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.104189 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.104496 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.104500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.105147 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:05.105354 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.105360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.106200 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.106202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.107084 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.107086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.107501 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.107502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.107653 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.108191 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:05.108200 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:05.109772 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:05.111950 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.111957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.111959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.112209 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.112451 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.112452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.112452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.112555 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.112849 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:05.112855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:05.112981 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.113076 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.113234 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":100,"threads_started":1}
I20260811 11:13:05.114050 14136 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:05.114202 14136 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:05.114392 14136 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:05.114830 13615 ts_manager.cc:194] Registered new tserver with Master: d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049)
I20260811 11:13:05.115209 13615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:46969
I20260811 11:13:05.122427 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:13:05.130632 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.123938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.29.145.62:38291)
0811 11:13:05.124009 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.124011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.124083 (+ 72us) server_negotiation.cc:407] Connection header received
0811 11:13:05.124159 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.124160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.124170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.124219 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:05.124842 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.125378 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.126216 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.126418 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.126479 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.127800 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:05.127801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:05.127802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:05.127809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:05.129471 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.129908 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.129909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.129910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.129965 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.130164 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.130165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.130166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.130319 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:05.130405 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.130497 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.130561 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:13:05.132361 13615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:13:05.134032 13615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:13:05.140975 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.136694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.29.145.3:45049)
0811 11:13:05.136860 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.136865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.137085 (+ 220us) server_negotiation.cc:407] Connection header received
0811 11:13:05.137192 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.137196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.137218 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.137307 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:05.137490 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.138401 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.139725 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.140524 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140595 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.140663 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.140848 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.140890 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260811 11:13:05.140978 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.136591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45049 (local address 127.0.0.1:34906)
0811 11:13:05.136963 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.136972 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.137029 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.137315 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.137319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.137322 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:05.137394 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.137397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.138507 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.138508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.139640 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.139642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.140627 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.140721 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.140737 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.140805 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.140868 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":101,"threads_started":1}
I20260811 11:13:05.141158 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.136737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.29.145.2:35623)
0811 11:13:05.137189 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.137193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.137204 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:13:05.137246 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.137248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.137262 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.137343 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:05.137541 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.138866 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140036 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.140624 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140730 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.140790 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.141022 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.141067 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":89,"threads_started":1}
I20260811 11:13:05.141422 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.136580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35623 (local address 127.0.0.1:48730)
0811 11:13:05.136889 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.136932 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.137013 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.137368 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.137369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.137372 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:05.137437 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.137440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.138989 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.138990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.139946 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.139948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.140734 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.140803 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.140807 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.140867 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.140910 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":82,"threads_started":1}
I20260811 11:13:05.141898 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.136836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35081 (local address 127.0.0.1:39810)
0811 11:13:05.137443 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.137453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.137504 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.137808 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.137809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.137811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:05.137870 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.137871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.139193 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.139194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.140460 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.140463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.141612 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.141614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.141693 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.141698 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.141756 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.141798 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":93,"threads_started":1}
I20260811 11:13:05.141938 14065 tablet_service.cc:1467] Processing CreateTablet for tablet 8d18e32a55c24a03b197bb41880db083 (DEFAULT_TABLE table=test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:05.142225 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.137037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.29.145.1:35081)
0811 11:13:05.137230 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.137234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.137563 (+ 329us) server_negotiation.cc:407] Connection header received
0811 11:13:05.137605 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.137607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.137621 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.137690 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:05.137941 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.139081 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.140574 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.141445 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.141518 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.141573 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.142090 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.142139 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260811 11:13:05.142160 13905 tablet_service.cc:1467] Processing CreateTablet for tablet 8d18e32a55c24a03b197bb41880db083 (DEFAULT_TABLE table=test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:05.142391 14065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d18e32a55c24a03b197bb41880db083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:05.142532 13905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d18e32a55c24a03b197bb41880db083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:05.142872 13758 tablet_service.cc:1467] Processing CreateTablet for tablet 8d18e32a55c24a03b197bb41880db083 (DEFAULT_TABLE table=test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:05.143217 13758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d18e32a55c24a03b197bb41880db083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:05.145242 14159 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Bootstrap starting.
I20260811 11:13:05.145995 14159 tablet_bootstrap.cc:654] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:05.146222 14161 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Bootstrap starting.
I20260811 11:13:05.146730 14159 log.cc:824] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:05.146760 14162 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap starting.
I20260811 11:13:05.147006 14161 tablet_bootstrap.cc:654] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:05.147495 14162 tablet_bootstrap.cc:654] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:05.147547 14161 log.cc:824] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:05.147614 14159 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: No bootstrap required, opened a new log
I20260811 11:13:05.147675 14159 ts_tablet_manager.cc:1397] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:13:05.148063 14162 log.cc:824] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:05.148972 14161 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: No bootstrap required, opened a new log
I20260811 11:13:05.149049 14161 ts_tablet_manager.cc:1397] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:13:05.149780 14159 raft_consensus.cc:348] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.149999 14159 raft_consensus.cc:374] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:05.150041 14159 raft_consensus.cc:729] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5812faeebc9b4edeb5450b55cfe93334, State: Initialized, Role: FOLLOWER
I20260811 11:13:05.150182 14159 consensus_queue.cc:260] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.150484 14159 ts_tablet_manager.cc:1428] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 11:13:05.150583 14162 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: No bootstrap required, opened a new log
I20260811 11:13:05.150610 13986 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
I20260811 11:13:05.150660 14162 ts_tablet_manager.cc:1397] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:13:05.151065 14161 raft_consensus.cc:348] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.151219 14161 raft_consensus.cc:374] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:05.151264 14161 raft_consensus.cc:729] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a11b2c04c643eabb36394e50688fb6, State: Initialized, Role: FOLLOWER
I20260811 11:13:05.151364 14161 consensus_queue.cc:260] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.151719 14161 ts_tablet_manager.cc:1428] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:13:05.152686 14162 raft_consensus.cc:348] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.152824 14162 raft_consensus.cc:374] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:05.152848 14162 raft_consensus.cc:729] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1026bf93f7274f919c9241a61f8c5fcf, State: Initialized, Role: FOLLOWER
I20260811 11:13:05.152931 14162 consensus_queue.cc:260] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.153162 14162 ts_tablet_manager.cc:1428] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:13:05.153187 13830 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
I20260811 11:13:05.153328 14136 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
W20260811 11:13:05.164865 13987 tablet.cc:2369] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:13:05.242365 13831 tablet.cc:2369] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:05.319098 14168 raft_consensus.cc:482] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:05.319415 14168 raft_consensus.cc:504] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.320097 14168 leader_election.cc:288] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623), 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081)
I20260811 11:13:05.324327 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.320581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:34041 (local address 127.29.145.2:35623)
0811 11:13:05.320726 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.320729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.320742 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:13:05.320786 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.320787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.320800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.320879 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:05.321208 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.322101 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.323407 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.323850 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.323948 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.323963 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.324160 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.324227 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 11:13:05.324309 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.320441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35623 (local address 127.29.145.3:34041)
0811 11:13:05.320579 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.320598 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.320684 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.320902 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.320903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.320905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:05.321092 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.321096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.322204 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.322205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.323297 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.323301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.323960 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.323962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.324073 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.324078 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.324131 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.324191 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 11:13:05.324324 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.320611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35081 (local address 127.29.145.3:43711)
0811 11:13:05.320833 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.320841 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.321029 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.321326 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.321327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.321330 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:05.321402 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.321403 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.322307 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.322308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.323346 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.323349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.324097 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.324098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.324171 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.324174 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.324227 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.324261 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":147,"threads_started":1}
I20260811 11:13:05.324396 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.320713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:43711 (local address 127.29.145.1:35081)
0811 11:13:05.320768 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.320771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.320917 (+ 146us) server_negotiation.cc:407] Connection header received
0811 11:13:05.321199 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.321201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.321213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.321280 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:05.321518 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.322237 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.323446 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.323924 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.324073 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.324084 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.324252 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.324304 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:13:05.324795 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d18e32a55c24a03b197bb41880db083" candidate_uuid: "d5a11b2c04c643eabb36394e50688fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1026bf93f7274f919c9241a61f8c5fcf" is_pre_election: true
I20260811 11:13:05.324846 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d18e32a55c24a03b197bb41880db083" candidate_uuid: "d5a11b2c04c643eabb36394e50688fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5812faeebc9b4edeb5450b55cfe93334" is_pre_election: true
I20260811 11:13:05.325062 13929 raft_consensus.cc:2454] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a11b2c04c643eabb36394e50688fb6 in term 0.
I20260811 11:13:05.325062 13779 raft_consensus.cc:2454] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5a11b2c04c643eabb36394e50688fb6 in term 0.
I20260811 11:13:05.325346 14015 leader_election.cc:302] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6; no voters:
I20260811 11:13:05.325505 14168 raft_consensus.cc:2781] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:13:05.325569 14168 raft_consensus.cc:482] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:13:05.325600 14168 raft_consensus.cc:3037] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:05.326529 14168 raft_consensus.cc:504] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.326699 14168 leader_election.cc:288] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [CANDIDATE]: Term 1 election: Requested vote from peers 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623), 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081)
I20260811 11:13:05.326956 13779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d18e32a55c24a03b197bb41880db083" candidate_uuid: "d5a11b2c04c643eabb36394e50688fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1026bf93f7274f919c9241a61f8c5fcf"
I20260811 11:13:05.326956 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d18e32a55c24a03b197bb41880db083" candidate_uuid: "d5a11b2c04c643eabb36394e50688fb6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5812faeebc9b4edeb5450b55cfe93334"
I20260811 11:13:05.327067 13779 raft_consensus.cc:3037] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:05.327068 13929 raft_consensus.cc:3037] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:05.328274 13929 raft_consensus.cc:2454] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a11b2c04c643eabb36394e50688fb6 in term 1.
I20260811 11:13:05.328274 13779 raft_consensus.cc:2454] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5a11b2c04c643eabb36394e50688fb6 in term 1.
I20260811 11:13:05.328518 14015 leader_election.cc:302] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf, d5a11b2c04c643eabb36394e50688fb6; no voters:
I20260811 11:13:05.328648 14168 raft_consensus.cc:2781] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:05.328886 14168 raft_consensus.cc:686] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 1 LEADER]: Becoming Leader. State: Replica: d5a11b2c04c643eabb36394e50688fb6, State: Running, Role: LEADER
I20260811 11:13:05.329025 14168 consensus_queue.cc:237] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.330245 13614 catalog_manager.cc:5495] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3). New cstate: current_term: 1 leader_uuid: "d5a11b2c04c643eabb36394e50688fb6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } health_report { overall_health: UNKNOWN } } }
W20260811 11:13:05.353186 14137 tablet.cc:2369] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:05.373449 14155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.370252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34916 (local address 127.29.145.3:45049)
0811 11:13:05.370329 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.370331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.370341 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:13:05.370391 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.370399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.370423 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.370524 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:05.370710 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.371320 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.372376 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.372619 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.372689 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.372920 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:05.373089 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:13:05.373179 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.373232 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.373263 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:13:05.376175 13929 raft_consensus.cc:1261] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Refusing update from remote peer d5a11b2c04c643eabb36394e50688fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:05.376175 13779 raft_consensus.cc:1261] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Refusing update from remote peer d5a11b2c04c643eabb36394e50688fb6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:05.376694 14168 consensus_queue.cc:1035] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:05.376893 14171 consensus_queue.cc:1035] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:05.380560 14176 mvcc.cc:205] Tried to move back new op lower bound from 7317286032896847872 to 7317286032708726784. Current Snapshot: MvccSnapshot[applied={T|T < 7317286032896847872}]
I20260811 11:13:05.380112 14174 mvcc.cc:205] Tried to move back new op lower bound from 7317286032896847872 to 7317286032708726784. Current Snapshot: MvccSnapshot[applied={T|T < 7317286032896847872}]
I20260811 11:13:05.381414 14175 mvcc.cc:205] Tried to move back new op lower bound from 7317286032896847872 to 7317286032708726784. Current Snapshot: MvccSnapshot[applied={T|T < 7317286032896847872}]
I20260811 11:13:05.384508 13614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:05.384683 13614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:05.385872 13614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Sending DeleteTablet for 3 replicas of tablet 8d18e32a55c24a03b197bb41880db083
I20260811 11:13:05.386322 13905 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d18e32a55c24a03b197bb41880db083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:05 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:48730
I20260811 11:13:05.386399 14065 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d18e32a55c24a03b197bb41880db083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:05 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:34906
I20260811 11:13:05.386430 13758 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d18e32a55c24a03b197bb41880db083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:05 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:39810
I20260811 11:13:05.386852 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13685
I20260811 11:13:05.394083 14187 tablet_replica.cc:330] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: stopping tablet replica
I20260811 11:13:05.394333 14187 raft_consensus.cc:2229] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:13:05.394600 14187 raft_consensus.cc:2258] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:05.395148 14186 tablet_replica.cc:330] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: stopping tablet replica
I20260811 11:13:05.395537 14186 raft_consensus.cc:2229] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:05.395881 14186 raft_consensus.cc:2258] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:05.396867 14186 ts_tablet_manager.cc:1905] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:05.399581 14186 ts_tablet_manager.cc:1918] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:05.399705 14186 log.cc:1196] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/8d18e32a55c24a03b197bb41880db083
I20260811 11:13:05.400051 14186 ts_tablet_manager.cc:1939] T 8d18e32a55c24a03b197bb41880db083 P 5812faeebc9b4edeb5450b55cfe93334: Deleting consensus metadata
I20260811 11:13:05.400765 13602 catalog_manager.cc:4841] TS 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): tablet 8d18e32a55c24a03b197bb41880db083 (table test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]) successfully deleted
I20260811 11:13:05.402609 14187 ts_tablet_manager.cc:1905] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:13:05.403548 13602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:35081 (error 108)
I20260811 11:13:05.404059 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13836
I20260811 11:13:05.406023 14187 ts_tablet_manager.cc:1918] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:05.406164 14187 log.cc:1196] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/8d18e32a55c24a03b197bb41880db083
I20260811 11:13:05.406497 14187 ts_tablet_manager.cc:1939] T 8d18e32a55c24a03b197bb41880db083 P d5a11b2c04c643eabb36394e50688fb6: Deleting consensus metadata
I20260811 11:13:05.407167 13600 catalog_manager.cc:4841] TS d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): tablet 8d18e32a55c24a03b197bb41880db083 (table test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]) successfully deleted
I20260811 11:13:05.410602 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.410404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35081 (local address 127.0.0.1:39824)
0811 11:13:05.410479 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.410547 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:13:05.410815 13602 catalog_manager.cc:4584] TS 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d18e32a55c24a03b197bb41880db083: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35081: connect: Connection refused (error 111)
I20260811 11:13:05.416427 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13995
I20260811 11:13:05.428880 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 13580
I20260811 11:13:05.440083 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--webserver_interface=127.29.145.62
--webserver_port=38669
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:38291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:05.535382 14190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:05.535660 14190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:05.535684 14190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:05.537027 14190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:05.537073 14190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:05.537088 14190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:05.537101 14190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:05.537113 14190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:05.538808 14190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:38291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=38669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:05.539122 14190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:05.539399 14190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:05.551133 14201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.551154 14197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.551137 14199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.551508 14190 server_base.cc:1034] running on GCE node
I20260811 11:13:05.551834 14190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:05.552217 14190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:05.553400 14190 hybrid_clock.cc:648] HybridClock initialized: now 1786446785553370 us; error 54 us; skew 500 ppm
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:05.556365 14190 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:05.556977 14190 webserver.cc:460] Webserver started at http://127.29.145.62:38669/ using document root <none> and password file <none>
I20260811 11:13:05.557261 14190 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:05.557356 14190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:05.559051 14190 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:05.559844 14208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.560034 14190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:05.560158 14190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "23b2a23d923ceae23f403df39bbe0feb"
server_key_iv: "3eb904c861648c05715c9be1be9374bc"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.560704 14190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:05.576344 14190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:05.579006 14190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:05.579202 14190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:05.584427 14190 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:38291
I20260811 11:13:05.584496 14268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:38291 every 8 connection(s)
I20260811 11:13:05.585005 14190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:05.587272 14269 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:13:05.587411 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14190
I20260811 11:13:05.587780 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:35081
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=40259
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:05.588207 14269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap starting.
I20260811 11:13:05.600773 14269 log.cc:824] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:05.603438 14269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:05.603924 14269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap complete.
I20260811 11:13:05.606774 14269 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:05.607044 14269 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Initialized, Role: FOLLOWER
I20260811 11:13:05.607250 14269 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:05.607322 14269 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:05.607391 14269 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:05.607443 14269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:13:05.608726 14269 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:05.608856 14269 leader_election.cc:302] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc1758b60c2c4638b30b3ad6dbef0bfd; no voters:
I20260811 11:13:05.609119 14269 leader_election.cc:288] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:13:05.609166 14273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:13:05.609311 14273 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 LEADER]: Becoming Leader. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Running, Role: LEADER
I20260811 11:13:05.609418 14273 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:05.609452 14269 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:05.609714 14274 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:05.609808 14274 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:05.610327 14275 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc1758b60c2c4638b30b3ad6dbef0bfd. Latest consensus state: current_term: 2 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:05.610443 14275 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:05.610340 14277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:05.611496 14277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:05.611608 14277 catalog_manager.cc:1259] Loaded cluster ID: 59d97f1bee204c199baafcfcb4a6c49f
I20260811 11:13:05.611630 14277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:13:05.611871 14277 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 11:13:05.616649 14277 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 11:13:05.619539 14277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:05.620095 14277 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Loaded TSK: 0
I20260811 11:13:05.620368 14277 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:13:05.687728 14271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:05.688011 14271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:05.688051 14271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:05.689280 14271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:05.689347 14271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:05.689400 14271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:13:05.691066 14271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:35081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=40259
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:05.691390 14271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:05.691671 14271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:05.697708 14300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.697702 14297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.698158 14271 server_base.cc:1034] running on GCE node
W20260811 11:13:05.697702 14298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.698448 14271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:05.698753 14271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:05.699931 14271 hybrid_clock.cc:648] HybridClock initialized: now 1786446785699902 us; error 52 us; skew 500 ppm
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:05.703287 14271 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:13:05.703846 14271 webserver.cc:460] Webserver started at http://127.29.145.1:40259/ using document root <none> and password file <none>
I20260811 11:13:05.704159 14271 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:05.704252 14271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:05.706115 14271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:05.707005 14310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.707231 14271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:05.707358 14271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "1026bf93f7274f919c9241a61f8c5fcf"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "010adc3f6399e0ad4eb74690f2f26c57"
server_key_iv: "fb5a2146baf69aed2eb522a5fadb1702"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.707834 14271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:05.737020 14271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:05.740080 14271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:05.740294 14271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:05.740615 14271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:05.741262 14318 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:13:05.743700 14271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:13:05.743803 14271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 11:13:05.743884 14271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:13:05.744752 14271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:13:05.744841 14271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:05.744881 14318 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap starting.
I20260811 11:13:05.754096 14271 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:35081
I20260811 11:13:05.754297 14441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:35081 every 8 connection(s)
I20260811 11:13:05.754648 14271 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:13:05.756579 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14271
I20260811 11:13:05.756954 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:35623
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=41329
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:05.758205 14318 log.cc:824] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:05.761426 14318 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:05.762004 14318 tablet_bootstrap.cc:492] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap complete.
I20260811 11:13:05.763451 14318 ts_tablet_manager.cc:1397] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.006s
I20260811 11:13:05.765395 14318 raft_consensus.cc:348] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.765799 14318 raft_consensus.cc:729] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1026bf93f7274f919c9241a61f8c5fcf, State: Initialized, Role: FOLLOWER
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:05.766042 14318 consensus_queue.cc:260] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:05.766434 14318 ts_tablet_manager.cc:1428] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent starting tablet: real 0.003s user 0.001s sys 0.004s
I20260811 11:13:05.771602 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.755642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:56297)
0811 11:13:05.755867 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.755921 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.756775 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.757645 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.757651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.758213 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:05.758542 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.758551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.760240 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.760242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.761700 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.761704 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.763311 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.763312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.763508 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.764331 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:05.764345 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:05.766656 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:05.769337 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.769348 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.769351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.769649 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.770102 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.770103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.770103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.770252 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.770774 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:05.770783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:05.770961 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.771087 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.771336 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":127,"threads_started":1}
I20260811 11:13:05.771602 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.755782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:56297 (local address 127.29.145.62:38291)
0811 11:13:05.756045 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.756054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.756104 (+ 50us) server_negotiation.cc:407] Connection header received
0811 11:13:05.757293 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.757309 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.757393 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.757597 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:05.758695 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.760087 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.761886 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.763024 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.763127 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.766807 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:05.766814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:05.766815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:05.766832 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:05.769196 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.769771 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.769784 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.769785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.770008 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.770340 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.770341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.770341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.770678 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:05.770791 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.771167 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.771336 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":153,"threads_started":1}
I20260811 11:13:05.772799 14442 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:05.772925 14442 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:05.773180 14442 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:05.774156 14226 ts_manager.cc:194] Registered new tserver with Master: 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081)
I20260811 11:13:05.775453 14226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:56297
I20260811 11:13:05.776610 14442 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:13:05.786473 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.774917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35081 (local address 127.0.0.1:39828)
0811 11:13:05.775167 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.775215 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.775306 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.776247 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.776254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.776683 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:05.776772 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.776776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.778323 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.778325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.779041 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.779043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.779754 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.779755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.779840 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.780497 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:05.780503 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:05.782325 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:05.784881 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.784894 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.784898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.785170 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.785469 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.785471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.785471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.785552 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.785995 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:05.786005 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:05.786078 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.786168 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.786337 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":143,"threads_started":1}
I20260811 11:13:05.786474 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.775794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.29.145.1:35081)
0811 11:13:05.776045 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.776052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.776064 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:13:05.776110 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.776112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.776130 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.776866 (+ 736us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:05.776940 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.778220 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.779125 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.779664 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.779745 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.782431 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:05.782445 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:05.782445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:05.782455 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:05.784785 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.785270 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.785274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.785275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.785354 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.785634 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.785635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.785635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.785913 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:05.786017 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.786236 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.786349 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":150,"threads_started":1}
I20260811 11:13:05.787134 14366 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d18e32a55c24a03b197bb41880db083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:05 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:39828
I20260811 11:13:05.787875 14451 tablet_replica.cc:330] stopping tablet replica
I20260811 11:13:05.788039 14451 raft_consensus.cc:2229] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:05.788121 14451 raft_consensus.cc:2258] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:05.788535 14451 ts_tablet_manager.cc:1905] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:05.790968 14451 ts_tablet_manager.cc:1918] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:05.791051 14451 log.cc:1196] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/8d18e32a55c24a03b197bb41880db083
I20260811 11:13:05.791354 14451 ts_tablet_manager.cc:1939] T 8d18e32a55c24a03b197bb41880db083 P 1026bf93f7274f919c9241a61f8c5fcf: Deleting consensus metadata
I20260811 11:13:05.792096 14212 catalog_manager.cc:4841] TS 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081): tablet 8d18e32a55c24a03b197bb41880db083 (table test-table [id=b8a7da3776ee474db298f6adc7f3d4f9]) successfully deleted
W20260811 11:13:05.866941 14447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:05.867257 14447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:05.867308 14447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:05.868753 14447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:05.868830 14447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:05.868891 14447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:13:05.870731 14447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:35623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=41329
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:05.871066 14447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:05.871372 14447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:05.877911 14456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.877936 14457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:05.877938 14459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:05.878218 14447 server_base.cc:1034] running on GCE node
I20260811 11:13:05.878449 14447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:05.878708 14447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:05.879858 14447 hybrid_clock.cc:648] HybridClock initialized: now 1786446785879835 us; error 57 us; skew 500 ppm
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:05.882859 14447 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:13:05.883426 14447 webserver.cc:460] Webserver started at http://127.29.145.2:41329/ using document root <none> and password file <none>
I20260811 11:13:05.883701 14447 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:05.883800 14447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:05.885459 14447 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:05.886281 14468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.886509 14447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:05.886651 14447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "5812faeebc9b4edeb5450b55cfe93334"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "dcd3eeba15b49d689e35cfe504f737d4"
server_key_iv: "a5e28fafdb1fe936d4bfe842012083ff"
server_key_version: "encryptionkey@0"
I20260811 11:13:05.887112 14447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:05.914019 14447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:05.916519 14447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:05.916718 14447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:05.917055 14447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:05.917443 14447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:05.917500 14447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.917542 14447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:05.917585 14447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:05.925169 14447 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35623
I20260811 11:13:05.925310 14590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35623 every 8 connection(s)
I20260811 11:13:05.925693 14447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:13:05.926178 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14447
I20260811 11:13:05.926481 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:45049
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=43703
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:13:05 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446785, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:05.938616 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.926694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:35075 (local address 127.29.145.62:38291)
0811 11:13:05.926756 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:05.926760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:05.927067 (+ 307us) server_negotiation.cc:407] Connection header received
0811 11:13:05.927720 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:05.927723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:05.927747 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:05.927803 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:05.929249 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.929927 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.931523 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.931773 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.931837 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.934930 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:05.934932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:05.934933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:05.934941 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:05.936730 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.937301 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.937303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.937304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.937375 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:05.937729 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:05.937730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:05.937730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:05.937935 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:05.938002 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:05.938420 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 11:13:05.938521 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:13:05.939070 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:05.926706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:35075)
0811 11:13:05.926894 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:05.926948 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:05.927587 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:05.927894 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:05.927899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:05.928765 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:05.929108 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.929115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.930043 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.930045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.931418 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:05.931423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:05.931878 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:05.931879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:05.932070 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:05.932727 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:05.932741 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:05.934798 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:05.936821 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.936830 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.936834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.937205 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.937458 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:05.937459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:05.937459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:05.937630 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:05.938010 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:05.938021 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:05.938250 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 11:13:05.938400 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 11:13:05.938628 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260811 11:13:05.939738 14591 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:05.939872 14591 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:05.940161 14591 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:05.940714 14226 ts_manager.cc:194] Registered new tserver with Master: 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
I20260811 11:13:05.941159 14226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:35075
W20260811 11:13:06.031316 14595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:06.031608 14595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:06.031638 14595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:06.032974 14595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:06.033031 14595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:06.033066 14595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:13:06.034780 14595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:45049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=43703
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:06.035100 14595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:06.035414 14595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:06.041375 14602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.041363 14604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.041363 14601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.041957 14595 server_base.cc:1034] running on GCE node
I20260811 11:13:06.042227 14595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:06.042495 14595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:06.043679 14595 hybrid_clock.cc:648] HybridClock initialized: now 1786446786043656 us; error 42 us; skew 500 ppm
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:06.046799 14595 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:13:06.047389 14595 webserver.cc:460] Webserver started at http://127.29.145.3:43703/ using document root <none> and password file <none>
I20260811 11:13:06.047698 14595 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:06.047802 14595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:06.049497 14595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:06.050565 14611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.050824 14595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:06.050953 14595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "d5a11b2c04c643eabb36394e50688fb6"
format_stamp: "Formatted at 2026-08-11 11:13:05 on dist-test-slave-69rg"
server_key: "923d75486745acb4c711560009c2762a"
server_key_iv: "924ca060145c9981154d97bd9e2613c4"
server_key_version: "encryptionkey@0"
I20260811 11:13:06.051419 14595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:06.063513 14595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:06.067709 14595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:06.067930 14595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:06.068274 14595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:06.068749 14595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:06.068804 14595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.068849 14595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:06.068872 14595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.079639 14595 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:45049
I20260811 11:13:06.079715 14739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:45049 every 8 connection(s)
I20260811 11:13:06.080438 14595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:13:06.085083 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14595
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:06.094203 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.085567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.29.145.62:38291)
0811 11:13:06.085753 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.085755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.085894 (+ 139us) server_negotiation.cc:407] Connection header received
0811 11:13:06.085996 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.085997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.086013 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.086097 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:06.086791 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.087567 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.088411 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.088716 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.088775 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.090380 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:06.090383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:06.090383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:06.090395 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:06.092420 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:06.092913 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:06.092915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:06.092916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:06.092988 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:06.093349 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:06.093350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:06.093351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:06.093495 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:06.093718 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.093953 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.094057 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
I20260811 11:13:06.094352 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.081604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59365 (local address 127.29.145.62:38291)
0811 11:13:06.081682 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.081685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.083128 (+ 1443us) server_negotiation.cc:407] Connection header received
0811 11:13:06.083175 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.083176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.083194 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.083273 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:06.084710 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.085538 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.087316 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.087617 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.087730 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.090832 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:06.090834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:06.090835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:06.090843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:06.092656 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:06.093205 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:06.093206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:06.093206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:06.093271 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:06.093585 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:06.093586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:06.093587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:06.093704 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:06.093754 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.094178 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.094259 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:13:06.095135 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.081610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:59365)
0811 11:13:06.081892 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.081953 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.082950 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.083381 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:06.083386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:06.084278 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:06.084561 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.084568 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.085659 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.085660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.087165 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.087168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.087742 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.087743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.087867 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.088541 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:06.088555 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:06.090718 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:06.092737 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:06.092746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:06.092749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:06.093114 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:06.093347 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:06.093348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:06.093348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:06.093496 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:06.093793 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:06.093800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:06.094010 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 11:13:06.094541 (+ 531us) client_negotiation.cc:240] Negotiation successful
0811 11:13:06.094827 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260811 11:13:06.095604 14740 heartbeater.cc:347] Connected to a master server at 127.29.145.62:38291
I20260811 11:13:06.095752 14740 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:06.095976 14740 heartbeater.cc:511] Master 127.29.145.62:38291 requested a full tablet report, sending...
I20260811 11:13:06.096450 14226 ts_manager.cc:194] Registered new tserver with Master: d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049)
I20260811 11:13:06.096843 14226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:59365
I20260811 11:13:06.106572 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:13:06.110291 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.107132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.29.145.62:38291)
0811 11:13:06.107193 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.107196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.107372 (+ 176us) server_negotiation.cc:407] Connection header received
0811 11:13:06.107430 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.107431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.107494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.107580 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:06.107783 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.108323 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.109204 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.109427 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.109504 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.109756 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:06.109984 (+ 228us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:13:06.110108 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.110130 (+ 22us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.110157 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:13:06.110773 14226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:13:06.112998 14226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:13:06.115924 14366 tablet_service.cc:1467] Processing CreateTablet for tablet 018a98ab2c5e4a3884c918263abafb73 (DEFAULT_TABLE table=test-table [id=07ff737805b94b968554d69b89ebedce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:06.116216 14366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 018a98ab2c5e4a3884c918263abafb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:06.117854 14318 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap starting.
I20260811 11:13:06.118439 14318 tablet_bootstrap.cc:654] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:06.119136 14318 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: No bootstrap required, opened a new log
I20260811 11:13:06.119215 14318 ts_tablet_manager.cc:1397] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.119678 14318 raft_consensus.cc:348] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.119788 14318 raft_consensus.cc:374] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:06.119859 14318 raft_consensus.cc:729] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1026bf93f7274f919c9241a61f8c5fcf, State: Initialized, Role: FOLLOWER
I20260811 11:13:06.119912 14318 consensus_queue.cc:260] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.120021 14318 ts_tablet_manager.cc:1428] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.120468 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.115392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35623 (local address 127.0.0.1:48732)
0811 11:13:06.115577 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.115586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.115630 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.116442 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:06.116444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:06.116447 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:06.116566 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.116568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.118115 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.118116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.119189 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.119191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.120110 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.120111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.120191 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.120219 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 11:13:06.120310 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 11:13:06.120389 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260811 11:13:06.120533 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.115793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.29.145.2:35623)
0811 11:13:06.116069 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.116073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.116085 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:13:06.116141 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.116143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.116242 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.116330 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:06.116648 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.117881 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.119301 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.120013 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.120087 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.120159 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.120349 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.120422 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":116,"threads_started":1}
I20260811 11:13:06.120783 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.115338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45049 (local address 127.0.0.1:34922)
0811 11:13:06.115429 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.115442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.115487 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.115998 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:06.115999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:06.116001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:06.116069 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.116071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.117703 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.117705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.119113 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.119116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.120474 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.120475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.120579 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.120585 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:13:06.120624 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 11:13:06.120696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 11:13:06.121187 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.115473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.29.145.3:45049)
0811 11:13:06.115759 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.115767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.115785 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:13:06.115827 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.115829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.115847 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.115955 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:06.116201 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.117566 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.119418 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.120358 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.120860 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.120926 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.120951 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.121026 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":163,"threads_started":1}
I20260811 11:13:06.121788 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 018a98ab2c5e4a3884c918263abafb73 (DEFAULT_TABLE table=test-table [id=07ff737805b94b968554d69b89ebedce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:06.121942 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 018a98ab2c5e4a3884c918263abafb73 (DEFAULT_TABLE table=test-table [id=07ff737805b94b968554d69b89ebedce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:06.122323 14668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 018a98ab2c5e4a3884c918263abafb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:06.122344 14522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 018a98ab2c5e4a3884c918263abafb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:06.125705 14751 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Bootstrap starting.
I20260811 11:13:06.125990 14752 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Bootstrap starting.
I20260811 11:13:06.126561 14751 tablet_bootstrap.cc:654] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:06.126701 14752 tablet_bootstrap.cc:654] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:06.127282 14751 log.cc:824] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:06.127295 14752 log.cc:824] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:06.128149 14752 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: No bootstrap required, opened a new log
I20260811 11:13:06.128226 14752 ts_tablet_manager.cc:1397] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:13:06.128350 14751 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: No bootstrap required, opened a new log
I20260811 11:13:06.128419 14751 ts_tablet_manager.cc:1397] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 11:13:06.130491 14751 raft_consensus.cc:348] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.130491 14752 raft_consensus.cc:348] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.130688 14751 raft_consensus.cc:374] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:06.130688 14752 raft_consensus.cc:374] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:06.130728 14751 raft_consensus.cc:729] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a11b2c04c643eabb36394e50688fb6, State: Initialized, Role: FOLLOWER
I20260811 11:13:06.130728 14752 raft_consensus.cc:729] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5812faeebc9b4edeb5450b55cfe93334, State: Initialized, Role: FOLLOWER
I20260811 11:13:06.130862 14752 consensus_queue.cc:260] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.130893 14751 consensus_queue.cc:260] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.131212 14752 ts_tablet_manager.cc:1428] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:13:06.131212 14751 ts_tablet_manager.cc:1428] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 11:13:06.131222 14740 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
I20260811 11:13:06.131385 14591 heartbeater.cc:503] Master 127.29.145.62:38291 was elected leader, sending a full tablet report...
W20260811 11:13:06.176504 14593 tablet.cc:2369] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:06.232360 14448 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:06.232579 14448 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.233011 14448 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
I20260811 11:13:06.237799 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.233696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:40945 (local address 127.29.145.2:35623)
0811 11:13:06.233782 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.233785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.233797 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:13:06.233855 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.233856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.233867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.233927 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:06.234084 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.234904 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.236641 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.237276 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.237378 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.237394 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.237650 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.237705 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:13:06.237808 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.233538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45171 (local address 127.29.145.3:45049)
0811 11:13:06.233693 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.233696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.233706 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:13:06.233750 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.233751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.233768 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.233851 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:06.234085 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.234904 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.236637 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.237316 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.237386 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.237396 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.237668 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.237722 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 11:13:06.237840 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.233313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35623 (local address 127.29.145.1:40945)
0811 11:13:06.233426 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.233445 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.233552 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.233941 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:06.233943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:06.233944 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:06.234000 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.234001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.235020 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.235022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.236514 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.236519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.237429 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.237430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.237529 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.237556 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:13:06.237607 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 11:13:06.237701 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":133}
I20260811 11:13:06.237843 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.233330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45049 (local address 127.29.145.1:45171)
0811 11:13:06.233528 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.233540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.233606 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.233858 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:06.233860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:06.233862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:06.233982 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.233985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.235020 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.235022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.236514 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:06.236519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.237449 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.237451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:06.237530 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.237560 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 11:13:06.237646 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:13:06.237701 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":76,"thread_start_us":109,"threads_started":1}
I20260811 11:13:06.238354 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "018a98ab2c5e4a3884c918263abafb73" candidate_uuid: "1026bf93f7274f919c9241a61f8c5fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a11b2c04c643eabb36394e50688fb6" is_pre_election: true
I20260811 11:13:06.238615 14692 raft_consensus.cc:2454] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1026bf93f7274f919c9241a61f8c5fcf in term 0.
I20260811 11:13:06.238586 14544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "018a98ab2c5e4a3884c918263abafb73" candidate_uuid: "1026bf93f7274f919c9241a61f8c5fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5812faeebc9b4edeb5450b55cfe93334" is_pre_election: true
I20260811 11:13:06.238807 14544 raft_consensus.cc:2454] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1026bf93f7274f919c9241a61f8c5fcf in term 0.
I20260811 11:13:06.238911 14313 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf, d5a11b2c04c643eabb36394e50688fb6; no voters:
I20260811 11:13:06.239116 14448 raft_consensus.cc:2781] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:13:06.239190 14448 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:13:06.239212 14448 raft_consensus.cc:3037] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:06.240226 14448 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.240396 14448 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 1 election: Requested vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
I20260811 11:13:06.240572 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "018a98ab2c5e4a3884c918263abafb73" candidate_uuid: "1026bf93f7274f919c9241a61f8c5fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5a11b2c04c643eabb36394e50688fb6"
I20260811 11:13:06.240638 14544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "018a98ab2c5e4a3884c918263abafb73" candidate_uuid: "1026bf93f7274f919c9241a61f8c5fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5812faeebc9b4edeb5450b55cfe93334"
I20260811 11:13:06.240679 14692 raft_consensus.cc:3037] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:06.240723 14544 raft_consensus.cc:3037] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:06.241539 14692 raft_consensus.cc:2454] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1026bf93f7274f919c9241a61f8c5fcf in term 1.
I20260811 11:13:06.241628 14544 raft_consensus.cc:2454] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1026bf93f7274f919c9241a61f8c5fcf in term 1.
I20260811 11:13:06.241780 14313 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf, d5a11b2c04c643eabb36394e50688fb6; no voters:
I20260811 11:13:06.241969 14448 raft_consensus.cc:2781] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:06.242219 14448 raft_consensus.cc:686] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 LEADER]: Becoming Leader. State: Replica: 1026bf93f7274f919c9241a61f8c5fcf, State: Running, Role: LEADER
I20260811 11:13:06.242316 14448 consensus_queue.cc:237] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.243304 14226 catalog_manager.cc:5495] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1). New cstate: current_term: 1 leader_uuid: "1026bf93f7274f919c9241a61f8c5fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } health_report { overall_health: HEALTHY } } }
W20260811 11:13:06.256007 14443 tablet.cc:2369] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:06.269757 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.267016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.29.145.1:35081)
0811 11:13:06.267113 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:06.267115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:06.267125 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:13:06.267183 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:06.267184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:06.267208 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:06.267288 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:06.267452 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.268011 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.268831 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:06.269058 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:06.269119 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:06.269331 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:06.269470 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:13:06.269532 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:06.269650 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 11:13:06.269675 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:13:06.272295 14544 raft_consensus.cc:1261] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Refusing update from remote peer 1026bf93f7274f919c9241a61f8c5fcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:06.272464 14692 raft_consensus.cc:1261] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Refusing update from remote peer 1026bf93f7274f919c9241a61f8c5fcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:06.272900 14758 consensus_queue.cc:1035] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:06.273084 14448 consensus_queue.cc:1035] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:06.276499 14761 mvcc.cc:205] Tried to move back new op lower bound from 7317286036566568960 to 7317286036449329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317286036566568960}]
I20260811 11:13:06.277015 14762 mvcc.cc:205] Tried to move back new op lower bound from 7317286036566568960 to 7317286036449329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317286036566568960}]
I20260811 11:13:06.277648 14763 mvcc.cc:205] Tried to move back new op lower bound from 7317286036566568960 to 7317286036449329152. Current Snapshot: MvccSnapshot[applied={T|T < 7317286036566568960}]
I20260811 11:13:06.279465 14226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:06.279584 14226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:06.280675 14226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Sending DeleteTablet for 3 replicas of tablet 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:06.281119 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 018a98ab2c5e4a3884c918263abafb73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:06 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:34922
I20260811 11:13:06.281193 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 14271
I20260811 11:13:06.281204 14522 tablet_service.cc:1514] Processing DeleteTablet for tablet 018a98ab2c5e4a3884c918263abafb73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:06 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:48732
I20260811 11:13:06.281575 14771 tablet_replica.cc:330] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: stopping tablet replica
I20260811 11:13:06.281697 14771 raft_consensus.cc:2229] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:06.282984 14771 raft_consensus.cc:2258] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:06.284125 14771 ts_tablet_manager.cc:1905] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:06.286482 14771 ts_tablet_manager.cc:1918] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:06.286662 14771 log.cc:1196] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:06.286916 14772 tablet_replica.cc:330] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: stopping tablet replica
I20260811 11:13:06.286971 14771 ts_tablet_manager.cc:1939] T 018a98ab2c5e4a3884c918263abafb73 P d5a11b2c04c643eabb36394e50688fb6: Deleting consensus metadata
I20260811 11:13:06.287096 14772 raft_consensus.cc:2229] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:06.287261 14772 raft_consensus.cc:2258] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:06.287492 14211 catalog_manager.cc:4841] TS d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): tablet 018a98ab2c5e4a3884c918263abafb73 (table test-table [id=07ff737805b94b968554d69b89ebedce]) successfully deleted
I20260811 11:13:06.288161 14772 ts_tablet_manager.cc:1905] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:06.289999 14772 ts_tablet_manager.cc:1918] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:06.290126 14772 log.cc:1196] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:06.290465 14772 ts_tablet_manager.cc:1939] T 018a98ab2c5e4a3884c918263abafb73 P 5812faeebc9b4edeb5450b55cfe93334: Deleting consensus metadata
I20260811 11:13:06.291149 14212 catalog_manager.cc:4841] TS 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): tablet 018a98ab2c5e4a3884c918263abafb73 (table test-table [id=07ff737805b94b968554d69b89ebedce]) successfully deleted
W20260811 11:13:06.294730 14212 connection.cc:535] client connection to 127.29.145.1:35081 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 11:13:06.294865 14212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:13:06.295344 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 14447
I20260811 11:13:06.295519 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.295319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:35081 (local address 127.0.0.1:39860)
0811 11:13:06.295400 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.295469 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 11:13:06.299336 14212 catalog_manager.cc:4584] TS 1026bf93f7274f919c9241a61f8c5fcf (127.29.145.1:35081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 018a98ab2c5e4a3884c918263abafb73: Network error: Client connection negotiation failed: client connection to 127.29.145.1:35081: connect: Connection refused (error 111)
I20260811 11:13:06.304999 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 14595
I20260811 11:13:06.312974 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 14190
I20260811 11:13:06.323066 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--webserver_interface=127.29.145.62
--webserver_port=38669
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:38291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:06.415511 14774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:06.415781 14774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:06.415820 14774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:06.416965 14774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:06.417038 14774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:06.417062 14774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:06.417085 14774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:06.417114 14774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:06.418664 14774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:38291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:38291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=38669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:06.418953 14774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:06.419201 14774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:06.424703 14780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.424695 14779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.424677 14783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.424942 14774 server_base.cc:1034] running on GCE node
I20260811 11:13:06.425220 14774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:06.425578 14774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:06.426779 14774 hybrid_clock.cc:648] HybridClock initialized: now 1786446786426725 us; error 103 us; skew 500 ppm
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:06.429667 14774 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:06.430217 14774 webserver.cc:460] Webserver started at http://127.29.145.62:38669/ using document root <none> and password file <none>
I20260811 11:13:06.430474 14774 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:06.430549 14774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:06.431969 14774 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:06.432724 14790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.432916 14774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:06.433027 14774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "23b2a23d923ceae23f403df39bbe0feb"
server_key_iv: "3eb904c861648c05715c9be1be9374bc"
server_key_version: "encryptionkey@0"
I20260811 11:13:06.433403 14774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:06.454576 14774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:06.457454 14774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:06.457612 14774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:06.463024 14774 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:38291
I20260811 11:13:06.463061 14851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:38291 every 8 connection(s)
I20260811 11:13:06.463498 14774 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:06.465485 14852 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:13:06.466332 14852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap starting.
I20260811 11:13:06.471201 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14774
I20260811 11:13:06.471585 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:35081
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=40259
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:06.477208 14852 log.cc:824] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:06.480882 14852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:06.481266 14852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd: Bootstrap complete.
I20260811 11:13:06.485450 14852 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:06.485910 14852 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Initialized, Role: FOLLOWER
I20260811 11:13:06.486088 14852 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:06.486160 14852 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:06.486199 14852 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:06.486246 14852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 2 FOLLOWER]: Advancing to term 3
I20260811 11:13:06.487625 14852 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:06.487740 14852 leader_election.cc:302] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc1758b60c2c4638b30b3ad6dbef0bfd; no voters:
I20260811 11:13:06.487905 14852 leader_election.cc:288] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 11:13:06.487993 14856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 3 FOLLOWER]: Leader election won for term 3
I20260811 11:13:06.488166 14852 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:06.488202 14856 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [term 3 LEADER]: Becoming Leader. State: Replica: dc1758b60c2c4638b30b3ad6dbef0bfd, State: Running, Role: LEADER
I20260811 11:13:06.488345 14856 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } }
I20260811 11:13:06.488706 14857 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:06.488783 14857 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:06.489012 14858 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc1758b60c2c4638b30b3ad6dbef0bfd. Latest consensus state: current_term: 3 leader_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc1758b60c2c4638b30b3ad6dbef0bfd" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 38291 } } }
I20260811 11:13:06.489080 14866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:06.489148 14858 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc1758b60c2c4638b30b3ad6dbef0bfd [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:06.490013 14866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:06.490130 14866 catalog_manager.cc:1259] Loaded cluster ID: 59d97f1bee204c199baafcfcb4a6c49f
I20260811 11:13:06.490336 14866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 11:13:06.490631 14866 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f61580e19ad google::LogMessage::Fail() at ??:0
@ 0x7f61580e488a google::LogMessage::SendToLog() at ??:0
@ 0x7f61580e14de google::LogMessage::Flush() at ??:0
@ 0x7f61580e2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f61600020f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f616000b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6159faf0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6159fa4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f615ae086db start_thread at ??:0
@ 0x7f6156bdf61f clone at ??:0
W20260811 11:13:06.597790 14854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:06.598101 14854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:06.598129 14854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:06.599321 14854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:06.599366 14854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:06.599395 14854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:13:06.600934 14854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:35081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=40259
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:06.601235 14854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:06.601493 14854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:06.607264 14891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.607324 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.607347 14892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.608626 14854 server_base.cc:1034] running on GCE node
I20260811 11:13:06.608880 14854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:06.609355 14854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:06.610641 14854 hybrid_clock.cc:648] HybridClock initialized: now 1786446786610575 us; error 86 us; skew 500 ppm
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:06.613480 14854 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:13:06.614090 14854 webserver.cc:460] Webserver started at http://127.29.145.1:40259/ using document root <none> and password file <none>
I20260811 11:13:06.614360 14854 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:06.614409 14854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:06.615900 14854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.616761 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.616974 14854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.617085 14854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "1026bf93f7274f919c9241a61f8c5fcf"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "010adc3f6399e0ad4eb74690f2f26c57"
server_key_iv: "fb5a2146baf69aed2eb522a5fadb1702"
server_key_version: "encryptionkey@0"
I20260811 11:13:06.617481 14854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:06.635042 14854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:06.637609 14854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:06.637868 14854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:06.638171 14854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:06.638685 14910 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:13:06.640774 14854 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:13:06.640846 14854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 11:13:06.640899 14854 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:13:06.641570 14854 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:13:06.641641 14854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:06.641708 14910 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap starting.
I20260811 11:13:06.649466 14854 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:35081
I20260811 11:13:06.649629 15033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:35081 every 8 connection(s)
I20260811 11:13:06.650043 14854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:13:06.652801 14910 log.cc:824] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:06.655663 14910 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:06.656018 14910 tablet_bootstrap.cc:492] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Bootstrap complete.
I20260811 11:13:06.657310 14910 ts_tablet_manager.cc:1397] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260811 11:13:06.658829 14910 raft_consensus.cc:348] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.659186 14910 raft_consensus.cc:729] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1026bf93f7274f919c9241a61f8c5fcf, State: Initialized, Role: FOLLOWER
I20260811 11:13:06.659337 14910 consensus_queue.cc:260] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:06.659649 14910 ts_tablet_manager.cc:1428] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
I20260811 11:13:06.660323 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 14854
I20260811 11:13:06.660619 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:35623
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=41329
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:06.741590 15036 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:13:06.650868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:54553)
0811 11:13:06.651065 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.651111 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:06.651587 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:06.741354 (+ 89767us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
W20260811 11:13:06.741822 15034 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38291 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:38291: Client connection negotiation failed: client connection to 127.29.145.62:38291: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:13:06.742208 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.742082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:59601)
0811 11:13:06.742140 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.742182 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:13:06.742630 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.742471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:56567)
0811 11:13:06.742542 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.742594 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:13:06.742866 15034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:13:06.761359 15039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:06.761574 15039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:06.761598 15039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:06.762821 15039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:06.762866 15039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:06.762893 15039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:13:06.764410 15039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:35623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=41329
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:06.764672 15039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:06.764920 15039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:06.771102 15045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.771102 15044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.771340 15047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.771878 15039 server_base.cc:1034] running on GCE node
I20260811 11:13:06.772112 15039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:06.772378 15039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:06.773558 15039 hybrid_clock.cc:648] HybridClock initialized: now 1786446786773545 us; error 30 us; skew 500 ppm
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:06.782004 15039 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:13:06.782546 15039 webserver.cc:460] Webserver started at http://127.29.145.2:41329/ using document root <none> and password file <none>
I20260811 11:13:06.782822 15039 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:06.782871 15039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:06.784351 15039 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:13:06.785056 15054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.785301 15039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.785431 15039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "5812faeebc9b4edeb5450b55cfe93334"
format_stamp: "Formatted at 2026-08-11 11:13:04 on dist-test-slave-69rg"
server_key: "dcd3eeba15b49d689e35cfe504f737d4"
server_key_iv: "a5e28fafdb1fe936d4bfe842012083ff"
server_key_version: "encryptionkey@0"
I20260811 11:13:06.785859 15039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:06.802409 15039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:06.804663 15039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:06.804850 15039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:06.805128 15039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:06.805495 15039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:06.805552 15039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.805594 15039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:06.805637 15039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.812101 15039 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35623
I20260811 11:13:06.812156 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35623 every 8 connection(s)
I20260811 11:13:06.812500 15039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:13:06.813594 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.813213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:48137)
0811 11:13:06.813369 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.813448 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
W20260811 11:13:06.814064 15182 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38291 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:38291: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
I20260811 11:13:06.814361 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.814263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:54189)
0811 11:13:06.814313 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.814342 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 11:13:06.814610 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.814557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:59939)
0811 11:13:06.814579 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.814598 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260811 11:13:06.814779 15182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:13:06.818063 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15039
I20260811 11:13:06.818364 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:45049
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:38291
--webserver_port=43703
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:06.903594 15185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:06.903806 15185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:06.903831 15185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:06.904968 15185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:06.905014 15185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:06.905040 15185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:13:06.906610 15185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:45049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=43703
--tserver_master_addrs=127.29.145.62:38291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:06.906869 15185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:06.907136 15185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:06.912737 15197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:06.912761 15192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.912925 15185 server_base.cc:1034] running on GCE node
W20260811 11:13:06.912761 15194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:06.913179 15185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:06.913441 15185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:06.914655 15185 hybrid_clock.cc:648] HybridClock initialized: now 1786446786914632 us; error 48 us; skew 500 ppm
Aug 11 11:13:06 dist-test-slave-69rg krb5kdc[13565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:06.917342 15185 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:13:06.917855 15185 webserver.cc:460] Webserver started at http://127.29.145.3:43703/ using document root <none> and password file <none>
I20260811 11:13:06.918066 15185 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:06.918126 15185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:06.919489 15185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.920181 15204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.920354 15185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:06.920462 15185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "d5a11b2c04c643eabb36394e50688fb6"
format_stamp: "Formatted at 2026-08-11 11:13:05 on dist-test-slave-69rg"
server_key: "923d75486745acb4c711560009c2762a"
server_key_iv: "924ca060145c9981154d97bd9e2613c4"
server_key_version: "encryptionkey@0"
I20260811 11:13:06.920864 15185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:06.944891 15185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:06.947295 15185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:06.947465 15185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:06.947768 15185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:06.948146 15185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:06.948201 15185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.948247 15185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:06.948278 15185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:06.955008 15185 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:45049
I20260811 11:13:06.955201 15336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:45049 every 8 connection(s)
I20260811 11:13:06.955407 15185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:13:06.956548 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.956140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:45337)
0811 11:13:06.956328 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.956409 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
W20260811 11:13:06.956943 15337 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:38291 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:38291: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
I20260811 11:13:06.957257 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.957146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:39881)
0811 11:13:06.957201 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.957239 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:13:06.957538 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:06.957450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:35429)
0811 11:13:06.957493 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:06.957515 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 11:13:06.957641 15337 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:13:06.965106 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15185
I20260811 11:13:07.743775 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:07.743398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:54983)
0811 11:13:07.743630 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:07.743684 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":129,"threads_started":1}
I20260811 11:13:07.815478 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:07.815194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:55075)
0811 11:13:07.815362 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:07.815405 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260811 11:13:07.958523 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:07.958143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:57739)
0811 11:13:07.958354 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:07.958418 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260811 11:13:08.374436 15345 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:08.374626 15345 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:08.375146 15345 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
Aug 11 11:13:08 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.2@KRBTEST.COM
Aug 11 11:13:08 dist-test-slave-69rg krb5kdc[13565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446786, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.3@KRBTEST.COM
I20260811 11:13:08.391955 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.375446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35623 (local address 127.29.145.1:46969)
0811 11:13:08.375680 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:08.375695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:08.375788 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:08.377209 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:08.377214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:08.377936 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:08.378294 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:08.378305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.380200 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.380201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:08.381765 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:08.381771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.382983 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.382984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:08.383184 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:08.384093 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:08.384107 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:08.386918 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:08.390001 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:08.390004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:08.390236 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:08.390672 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:08.390673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:08.390799 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:08.391410 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:08.391411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:08.391488 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 11:13:08.391589 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 11:13:08.391842 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":119,"thread_start_us":72,"threads_started":1}
I20260811 11:13:08.391971 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.375673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:46969 (local address 127.29.145.2:35623)
0811 11:13:08.375935 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:08.375943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:08.375969 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:13:08.376791 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:08.376796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:08.376890 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:08.377116 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:08.378472 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.380011 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.381990 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.382783 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.382964 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:08.387143 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:08.387149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:08.387149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:08.387165 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:08.389806 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390351 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:08.390356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:08.390356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:08.390471 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390916 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:08.390918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:08.390919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:08.391317 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:08.391431 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:08.391663 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 11:13:08.391833 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260811 11:13:08.392163 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.375642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:41727 (local address 127.29.145.3:45049)
0811 11:13:08.375877 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:08.375893 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:08.375952 (+ 59us) server_negotiation.cc:407] Connection header received
0811 11:13:08.376791 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:08.376796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:08.376881 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:08.377116 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:08.378494 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.380012 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.381990 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.382783 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.382951 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:08.387138 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:08.387145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:08.387146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:08.387163 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:08.389417 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390024 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:08.390028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:08.390028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:08.390142 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390598 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:08.390600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:08.390601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:08.391156 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:08.391335 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:08.391671 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 11:13:08.391862 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":79,"threads_started":1}
I20260811 11:13:08.392345 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.375446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:45049 (local address 127.29.145.1:41727)
0811 11:13:08.375633 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:08.375651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:08.375741 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:08.377209 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:08.377214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:08.377936 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:08.378294 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:08.378305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.380192 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.380195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:08.381765 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:08.381771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:08.382982 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:08.382984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:08.383184 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:08.384093 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:08.384107 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:08.386928 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:08.389527 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:08.389538 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:08.389541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:08.389909 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:08.390226 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:08.390228 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:08.390228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:08.390363 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:08.391304 (+ 941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:08.391312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:08.391478 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 11:13:08.391590 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:13:08.392062 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":322,"thread_start_us":79,"threads_started":1}
W20260811 11:13:08.393146 14905 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
W20260811 11:13:08.393359 14906 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:08.393452 14906 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf; no voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6
I20260811 11:13:08.393587 15345 raft_consensus.cc:2726] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:13:08.744566 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.744402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:49177)
0811 11:13:08.744482 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:08.744522 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 11:13:08.816483 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.816171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:58617)
0811 11:13:08.816376 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:08.816428 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260811 11:13:08.959512 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:08.959283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:37245)
0811 11:13:08.959426 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:08.959472 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260811 11:13:09.745728 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:09.745375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:41951)
0811 11:13:09.745586 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:09.745634 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":128,"threads_started":1}
I20260811 11:13:09.817539 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:09.817221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:44477)
0811 11:13:09.817412 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:09.817471 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260811 11:13:09.960562 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:09.960206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:38157)
0811 11:13:09.960411 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:09.960469 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":98,"threads_started":1}
I20260811 11:13:10.425025 15356 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:10.425264 15356 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:10.425819 15356 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
W20260811 11:13:10.426470 14905 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
W20260811 11:13:10.426594 14906 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:10.426630 14905 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf; no voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6
I20260811 11:13:10.426738 15356 raft_consensus.cc:2726] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:13:10.746847 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:10.746474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:43929)
0811 11:13:10.746710 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:10.746776 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":93,"threads_started":1}
I20260811 11:13:10.818639 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:10.818274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:46473)
0811 11:13:10.818465 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:10.818550 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260811 11:13:10.961510 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:10.961181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:35473)
0811 11:13:10.961350 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:10.961396 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260811 11:13:11.748023 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:11.747709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:38495)
0811 11:13:11.747901 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:11.747954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260811 11:13:11.819564 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:11.819235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:49671)
0811 11:13:11.819439 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:11.819482 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260811 11:13:11.962415 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:11.962111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:35703)
0811 11:13:11.962298 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:11.962347 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260811 11:13:12.748953 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:12.748673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:59185)
0811 11:13:12.748847 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:12.748881 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260811 11:13:12.820748 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:12.820365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:59969)
0811 11:13:12.820618 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:12.820677 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":156,"threads_started":1}
I20260811 11:13:12.963335 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:12.963041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:38015)
0811 11:13:12.963220 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:12.963259 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":108,"threads_started":1}
I20260811 11:13:13.362200 15367 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:13.362358 15367 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:13.362664 15367 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
W20260811 11:13:13.363158 14905 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
W20260811 11:13:13.363237 14906 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:13.363317 14906 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf; no voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6
I20260811 11:13:13.363392 15367 raft_consensus.cc:2726] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:13:13.750159 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:13.749616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:56907)
0811 11:13:13.749946 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:13.750085 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":106,"threads_started":1}
I20260811 11:13:13.822022 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:13.821543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:47563)
0811 11:13:13.821880 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:13.821958 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":95,"threads_started":1}
I20260811 11:13:13.964432 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:13.964099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:58273)
0811 11:13:13.964324 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:13.964373 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":109,"threads_started":1}
I20260811 11:13:14.751412 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:14.751138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:54795)
0811 11:13:14.751307 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:14.751345 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260811 11:13:14.823215 15372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:14.822815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:55349)
0811 11:13:14.823065 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:14.823146 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":121,"threads_started":1}
I20260811 11:13:14.965471 15373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:14.965136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:48297)
0811 11:13:14.965343 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:14.965403 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260811 11:13:15.752447 15374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:15.752141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:44153)
0811 11:13:15.752321 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:15.752378 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260811 11:13:15.824796 15375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:15.824117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:32773)
0811 11:13:15.824653 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:15.824712 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":186,"threads_started":1}
I20260811 11:13:15.971630 15376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:15.971197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:60255)
0811 11:13:15.971434 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:15.971496 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":102,"threads_started":1}
I20260811 11:13:16.753407 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:16.753103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:58821)
0811 11:13:16.753292 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:16.753337 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":89,"threads_started":1}
I20260811 11:13:16.825773 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:16.825453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:48195)
0811 11:13:16.825661 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:16.825700 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
I20260811 11:13:16.972760 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:16.972438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:56097)
0811 11:13:16.972641 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:16.972693 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":108,"threads_started":1}
I20260811 11:13:17.754632 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:17.754188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:57903)
0811 11:13:17.754505 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:17.754563 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":119,"threads_started":1}
I20260811 11:13:17.826879 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:17.826487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:36249)
0811 11:13:17.826768 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:17.826823 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":100,"threads_started":1}
I20260811 11:13:17.973757 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:17.973415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:45855)
0811 11:13:17.973637 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:17.973690 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":125,"threads_started":1}
I20260811 11:13:18.053705 15383 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:18.053915 15383 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:18.054222 15383 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
W20260811 11:13:18.054760 14906 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
W20260811 11:13:18.054883 14905 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:18.054961 14905 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf; no voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6
I20260811 11:13:18.055051 15383 raft_consensus.cc:2726] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:13:18.755743 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:18.755346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:39481)
0811 11:13:18.755623 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:18.755675 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":139,"threads_started":1}
I20260811 11:13:18.827948 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:18.827606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:35103)
0811 11:13:18.827826 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:18.827879 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":104,"threads_started":1}
I20260811 11:13:18.974748 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:18.974400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:33497)
0811 11:13:18.974631 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:18.974680 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":116,"threads_started":1}
I20260811 11:13:19.756685 15388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:19.756340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:37861)
0811 11:13:19.756568 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:19.756619 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":102,"threads_started":1}
I20260811 11:13:19.828927 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:19.828663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:43305)
0811 11:13:19.828818 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:19.828858 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260811 11:13:19.975772 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:19.975465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:33789)
0811 11:13:19.975653 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:19.975705 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260811 11:13:19.994750 15391 raft_consensus.cc:482] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:19.994907 15391 raft_consensus.cc:504] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5a11b2c04c643eabb36394e50688fb6" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 45049 } } peers { permanent_uuid: "5812faeebc9b4edeb5450b55cfe93334" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35623 } } peers { permanent_uuid: "1026bf93f7274f919c9241a61f8c5fcf" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 35081 } }
I20260811 11:13:19.995193 15391 leader_election.cc:288] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049), 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623)
W20260811 11:13:19.995729 14905 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d5a11b2c04c643eabb36394e50688fb6 (127.29.145.3:45049): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
W20260811 11:13:19.995828 14906 leader_election.cc:341] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5812faeebc9b4edeb5450b55cfe93334 (127.29.145.2:35623): Not found: Tablet not found: 018a98ab2c5e4a3884c918263abafb73
I20260811 11:13:19.995906 14906 leader_election.cc:302] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1026bf93f7274f919c9241a61f8c5fcf; no voters: 5812faeebc9b4edeb5450b55cfe93334, d5a11b2c04c643eabb36394e50688fb6
I20260811 11:13:19.996016 15391 raft_consensus.cc:2726] T 018a98ab2c5e4a3884c918263abafb73 P 1026bf93f7274f919c9241a61f8c5fcf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:13:20.757761 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:20.757437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:38601)
0811 11:13:20.757636 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:20.757692 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
I20260811 11:13:20.829927 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:20.829566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:50303)
0811 11:13:20.829765 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:20.829839 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260811 11:13:20.976794 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:20.976454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:46737)
0811 11:13:20.976669 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:20.976724 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":129,"threads_started":1}
I20260811 11:13:21.758883 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:21.758595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.1:55191)
0811 11:13:21.758769 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:21.758820 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260811 11:13:21.831034 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:21.830667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.2:44993)
0811 11:13:21.830855 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:21.830906 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260811 11:13:21.965579 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 14854
I20260811 11:13:21.973441 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15039
I20260811 11:13:21.977905 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:21.977544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:38291 (local address 127.29.145.3:57497)
0811 11:13:21.977726 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:21.977783 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:38291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260811 11:13:21.984207 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15185
2026-08-11T11:13:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18869 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:13:22 dist-test-slave-69rg krb5kdc[15403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:13:22 dist-test-slave-69rg krb5kdc[15403](info): set up 2 sockets
Aug 11 11:13:22 dist-test-slave-69rg krb5kdc[15403](info): commencing operation
krb5kdc: starting...
W20260811 11:13:23.136736 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:13:23.274420 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:23.367873 15419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:23.368122 15419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:23.368160 15419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:23.369313 15419 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:13:23.369385 15419 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:13:23.369408 15419 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:13:23.369428 15419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:23.369459 15419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:23.369484 15419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:23.369514 15419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:23.369541 15419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:23.371111 15419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:23.371424 15419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:23.371673 15419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:23.377691 15426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.377698 15425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.377720 15419 server_base.cc:1034] running on GCE node
W20260811 11:13:23.377703 15428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.378371 15419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:23.378681 15419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:23.379839 15419 hybrid_clock.cc:648] HybridClock initialized: now 1786446803379825 us; error 32 us; skew 500 ppm
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:23.382573 15419 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:23.383038 15419 webserver.cc:460] Webserver started at http://127.29.145.62:41869/ using document root <none> and password file <none>
I20260811 11:13:23.383229 15419 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:23.383270 15419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:23.383373 15419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:23.384402 15419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "95f02978e6e94ba88869a4459808f9af"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "c51ad1ae9f40d699c40d8329ba500f20"
server_key_iv: "a1b167597ab9debf5ba909520607f742"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.384709 15419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "95f02978e6e94ba88869a4459808f9af"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "c51ad1ae9f40d699c40d8329ba500f20"
server_key_iv: "a1b167597ab9debf5ba909520607f742"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.386148 15419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:13:23.386932 15435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.387189 15419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:23.387319 15419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "95f02978e6e94ba88869a4459808f9af"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "c51ad1ae9f40d699c40d8329ba500f20"
server_key_iv: "a1b167597ab9debf5ba909520607f742"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.387387 15419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:23.397318 15419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:23.399693 15419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:23.399852 15419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:23.404114 15419 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37699
I20260811 11:13:23.404208 15494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37699 every 8 connection(s)
I20260811 11:13:23.404546 15419 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:23.405498 15495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:23.407953 15495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap starting.
I20260811 11:13:23.408528 15495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:23.408800 15495 log.cc:824] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:23.409495 15495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: No bootstrap required, opened a new log
I20260811 11:13:23.410642 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15419
I20260811 11:13:23.410777 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:13:23.410984 30276 external_mini_cluster.cc:1428] Setting key ef30fb84b56afcb3ee27a903907a250a
I20260811 11:13:23.411427 15495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:23.411662 15495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:23.411751 15495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Initialized, Role: FOLLOWER
I20260811 11:13:23.411911 15495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:23.412001 15495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:23.412034 15495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:23.412083 15495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:23.413203 15495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:23.413318 15495 leader_election.cc:302] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f02978e6e94ba88869a4459808f9af; no voters:
I20260811 11:13:23.413497 15495 leader_election.cc:288] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:13:23.413661 15500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:23.413757 15495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:23.413861 15500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 LEADER]: Becoming Leader. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Running, Role: LEADER
I20260811 11:13:23.413982 15500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:23.414417 15500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:13:23.414498 15500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:23.414654 15500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f02978e6e94ba88869a4459808f9af. Latest consensus state: current_term: 1 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:13:23.414777 15500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
W20260811 11:13:23.415693 15515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:13:23.415767 15515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:13:23.415812 15509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:23.415975 15509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:23.417541 15509 catalog_manager.cc:1347] Generated new cluster ID: 65fbfe6fcc6c47e4a03e1cd46b0835af
I20260811 11:13:23.417594 15509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:23.424072 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.411587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.29.145.62:37699)
0811 11:13:23.411861 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.411872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.411904 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:13:23.412024 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.412032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.412106 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.412293 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:23.412971 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.414480 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.415672 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.416319 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.416428 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.419589 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:23.419593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:23.419593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:23.419605 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:23.422217 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.422710 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.422714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.422715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.422836 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.423085 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.423086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.423087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.423411 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:23.423567 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.423709 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.423846 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":149,"threads_started":1}
I20260811 11:13:23.438784 15509 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:13:23.439580 15509 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:23.446266 15509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Generated new TSK 0
I20260811 11:13:23.446487 15509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:13:23.488951 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:23.577970 15523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:23.578190 15523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:23.578214 15523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:23.579360 15523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:23.579401 15523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:23.579427 15523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:13:23.580865 15523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:23.581096 15523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:23.581337 15523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:23.586419 15528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.586411 15531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.586411 15529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.586714 15523 server_base.cc:1034] running on GCE node
I20260811 11:13:23.586934 15523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:23.587193 15523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:23.588352 15523 hybrid_clock.cc:648] HybridClock initialized: now 1786446803588340 us; error 29 us; skew 500 ppm
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:23.590996 15523 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:13:23.591476 15523 webserver.cc:460] Webserver started at http://127.29.145.1:38005/ using document root <none> and password file <none>
I20260811 11:13:23.591671 15523 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:23.591720 15523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:23.591812 15523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:23.592869 15523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "606d3394d0c8264c13f9a1bd33094267"
server_key_iv: "c2d07b7c15a772e46abd151d1c28ce1d"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.593204 15523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "606d3394d0c8264c13f9a1bd33094267"
server_key_iv: "c2d07b7c15a772e46abd151d1c28ce1d"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.594736 15523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:13:23.595561 15539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.595887 15523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:23.595979 15523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "606d3394d0c8264c13f9a1bd33094267"
server_key_iv: "c2d07b7c15a772e46abd151d1c28ce1d"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.596084 15523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:23.618587 15523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:23.620846 15523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:23.621012 15523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:23.621289 15523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:23.621610 15523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:23.621644 15523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.621667 15523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:23.621712 15523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.627985 15523 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44569
I20260811 11:13:23.628022 15665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44569 every 8 connection(s)
I20260811 11:13:23.628368 15523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:23.635847 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15523
I20260811 11:13:23.635968 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:13:23.636204 30276 external_mini_cluster.cc:1428] Setting key 4a4719befae20c6639d38b971923684d
I20260811 11:13:23.638307 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.629170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45519 (local address 127.29.145.62:37699)
0811 11:13:23.629240 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.629242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.629458 (+ 216us) server_negotiation.cc:407] Connection header received
0811 11:13:23.630097 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.630098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.630113 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.630169 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:23.631137 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.631662 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.632684 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.632912 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.633001 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.635160 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:23.635162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:23.635162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:23.635170 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:23.636763 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.637200 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.637201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.637202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.637267 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.637508 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.637509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.637509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.637687 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:23.637759 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.638119 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.638211 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:13:23.638477 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.629193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:45519)
0811 11:13:23.629337 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.629374 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.629976 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.630244 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.630248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.630826 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:23.631016 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.631020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.631778 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.631779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.632587 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.632589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.633015 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.633016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.633138 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.633633 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:23.633642 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:23.635059 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:23.636856 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.636863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.636866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.637119 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.637342 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.637343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.637343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.637442 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.637758 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:23.637764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:23.637946 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.638065 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.638235 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260811 11:13:23.638944 15666 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:23.639063 15666 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:23.639242 15666 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:23.639772 15453 ts_manager.cc:194] Registered new tserver with Master: f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569)
I20260811 11:13:23.640381 15453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:45519
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:13:23.666034 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:23.754410 15671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:23.754624 15671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:23.754648 15671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:23.755738 15671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:23.755779 15671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:23.755805 15671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:13:23.757200 15671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:23.757398 15671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:23.757591 15671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:23.763168 15680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.763322 15678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.763474 15671 server_base.cc:1034] running on GCE node
W20260811 11:13:23.763291 15677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.763808 15671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:23.764096 15671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:23.765286 15671 hybrid_clock.cc:648] HybridClock initialized: now 1786446803765282 us; error 41 us; skew 500 ppm
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:23.768359 15671 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:13:23.768821 15671 webserver.cc:460] Webserver started at http://127.29.145.2:39839/ using document root <none> and password file <none>
I20260811 11:13:23.769091 15671 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:23.769138 15671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:23.769321 15671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:23.770464 15671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "31d94e62a094409e8e18f0548489e3a8"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "02f4d32592f810078d1ac126bb27b518"
server_key_iv: "e4f885ca95b08ce5d747b700a4bee7c5"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.770831 15671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "31d94e62a094409e8e18f0548489e3a8"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "02f4d32592f810078d1ac126bb27b518"
server_key_iv: "e4f885ca95b08ce5d747b700a4bee7c5"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.772406 15671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:13:23.773216 15689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.773469 15671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:23.773598 15671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "31d94e62a094409e8e18f0548489e3a8"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "02f4d32592f810078d1ac126bb27b518"
server_key_iv: "e4f885ca95b08ce5d747b700a4bee7c5"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.773672 15671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:23.783377 15671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:23.786067 15671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:23.786418 15671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:23.786791 15671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:23.787177 15671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:23.787245 15671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.787290 15671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:23.787323 15671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.794662 15671 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36239
I20260811 11:13:23.794826 15816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36239 every 8 connection(s)
I20260811 11:13:23.795141 15671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:23.801966 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15671
I20260811 11:13:23.802121 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:13:23.802333 30276 external_mini_cluster.cc:1428] Setting key 28def90fb8d23a2da730eb0c910d9f32
I20260811 11:13:23.805431 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.796065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:39445 (local address 127.29.145.62:37699)
0811 11:13:23.796173 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.796175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.796367 (+ 192us) server_negotiation.cc:407] Connection header received
0811 11:13:23.796857 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.796858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.796868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.796932 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:23.797914 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.798456 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.799625 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.799821 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.799891 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.802443 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:23.802445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:23.802445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:23.802451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:23.803990 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.804434 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.804435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.804436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.804500 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.804766 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.804767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.804768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.804886 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:23.804961 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.805250 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.805340 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 11:13:23.805624 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.796026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:39445)
0811 11:13:23.796230 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.796275 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.796774 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.796984 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.796988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.797584 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:23.797787 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.797792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.798561 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.798562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.799544 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.799547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.799925 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.799926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.800078 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.800712 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:23.800721 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:23.802315 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:23.804103 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.804110 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.804112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.804350 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.804580 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.804581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.804582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.804696 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.804966 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:23.804973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:23.805118 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.805228 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.805388 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":125,"threads_started":1}
I20260811 11:13:23.806319 15818 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:23.806425 15818 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:23.806618 15818 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:23.807029 15453 ts_manager.cc:194] Registered new tserver with Master: 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239)
I20260811 11:13:23.807524 15453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:39445
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:13:23.832247 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:23.921362 15824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:23.921618 15824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:23.921655 15824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:23.922870 15824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:23.922942 15824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:23.922988 15824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:13:23.924566 15824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:23.924866 15824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:23.925122 15824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:23.931025 15830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.931025 15833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:23.931159 15829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:23.931380 15824 server_base.cc:1034] running on GCE node
I20260811 11:13:23.931627 15824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:23.931906 15824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:23.933090 15824 hybrid_clock.cc:648] HybridClock initialized: now 1786446803933075 us; error 43 us; skew 500 ppm
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:23.935938 15824 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:13:23.936470 15824 webserver.cc:460] Webserver started at http://127.29.145.3:45269/ using document root <none> and password file <none>
I20260811 11:13:23.936733 15824 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:23.936805 15824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:23.936965 15824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:13:23.938184 15824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "1c0c9f1ebb919a0bdf0ea34ca1623f91"
server_key_iv: "197ea9ab9f1f06d89a2587fd16dd0fc2"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.938593 15824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "1c0c9f1ebb919a0bdf0ea34ca1623f91"
server_key_iv: "197ea9ab9f1f06d89a2587fd16dd0fc2"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.940109 15824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:13:23.940976 15841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.941196 15824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:23.941339 15824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "1c0c9f1ebb919a0bdf0ea34ca1623f91"
server_key_iv: "197ea9ab9f1f06d89a2587fd16dd0fc2"
server_key_version: "encryptionkey@0"
I20260811 11:13:23.941424 15824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:23.958026 15824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:23.960333 15824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:23.960497 15824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:23.960743 15824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:23.961062 15824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:23.961124 15824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.961169 15824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:23.961198 15824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:23.967408 15824 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:46813
I20260811 11:13:23.967450 15976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:46813 every 8 connection(s)
I20260811 11:13:23.967774 15824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:13:23.968479 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 15824
I20260811 11:13:23.968564 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:13:23.968782 30276 external_mini_cluster.cc:1428] Setting key 3626b53491bbb021f52489668b4815bb
Aug 11 11:13:23 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446803, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:23.977849 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.968500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:49857 (local address 127.29.145.62:37699)
0811 11:13:23.968545 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.968547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.968784 (+ 237us) server_negotiation.cc:407] Connection header received
0811 11:13:23.969674 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.969674 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.969685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.969736 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:23.970798 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.971330 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.972313 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.972504 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.972562 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.974766 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:23.974767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:23.974767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:23.974774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:23.976210 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.976707 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.976709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.976709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.976784 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.977064 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.977065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.977065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.977171 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:23.977224 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.977602 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.977724 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:13:23.977994 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.968483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:49857)
0811 11:13:23.968645 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.968684 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.969535 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.969854 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.969858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.970505 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:23.970701 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.970706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.971421 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.971422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.972224 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.972226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.972580 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.972581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.972689 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.973268 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:23.973285 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:23.974666 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:23.976348 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.976356 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.976358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.976632 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.976870 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:23.976871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:23.976871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:23.976990 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:23.977250 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:23.977257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:23.977409 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.977558 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.977729 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260811 11:13:23.978514 15977 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:23.978627 15977 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:23.978835 15977 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:23.979195 15453 ts_manager.cc:194] Registered new tserver with Master: e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
I20260811 11:13:23.979589 15453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:49857
I20260811 11:13:23.980883 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:13:23.988082 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.982356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46224 (local address 127.29.145.62:37699)
0811 11:13:23.982421 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.982423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.982529 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:13:23.982592 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.982592 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.982601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.982650 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:23.983170 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.983679 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.984420 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.984599 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.984685 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.985783 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:23.985784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:23.985784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:23.985790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:23.986990 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.987331 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.987332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.987332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.987383 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:23.987607 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:23.987609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:23.987609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:23.987765 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:23.987855 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.987953 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.988016 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:13:23.989718 15453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:13:23.991534 15453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:13:23.998368 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.29.145.2:36239)
0811 11:13:23.994684 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.994688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.994700 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:13:23.994811 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.994813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.994832 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.995157 (+ 325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:23.995231 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.995908 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.997241 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.997937 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.998013 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.998070 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.998242 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.998283 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
I20260811 11:13:23.998360 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36239 (local address 127.0.0.1:42152)
0811 11:13:23.994337 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.994381 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.994445 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.994950 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.994953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.994955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:23.995041 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.995043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.996013 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.996014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.997156 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.997159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.998016 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.998017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.998122 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.998137 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.998212 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.998260 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260811 11:13:23.999140 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44569 (local address 127.0.0.1:49674)
0811 11:13:23.994555 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.994564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.994602 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.995003 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.995004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.995005 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:23.995061 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.995063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.996438 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.996439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.997553 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.997555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.998892 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.998893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.998965 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.998969 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.999031 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.999070 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":142,"threads_started":1}
I20260811 11:13:23.999334 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.29.145.1:44569)
0811 11:13:23.994854 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.994860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.994870 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:13:23.994908 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.994910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.994926 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.994995 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:23.995152 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.996328 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.997668 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.998517 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.998637 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.998695 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:23.999080 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 11:13:23.999125 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":100,"threads_started":1}
I20260811 11:13:23.999504 15742 tablet_service.cc:1467] Processing CreateTablet for tablet 30558408de574c86a64b6f50bd59c3aa (DEFAULT_TABLE table=test-table [id=8ec15c488a0145d3af26b512932122dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:23.999804 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:46813 (local address 127.0.0.1:58452)
0811 11:13:23.994671 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:23.994680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:23.994730 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:23.994949 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:23.994952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:23.994955 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:23.995113 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.995115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.997273 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.997274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.998292 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:23.998295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.999526 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.999527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:23.999593 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.999598 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:13:23.999631 (+ 33us) client_negotiation.cc:240] Negotiation successful
0811 11:13:23.999669 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":110,"thread_start_us":93,"threads_started":1}
I20260811 11:13:23.999917 15742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30558408de574c86a64b6f50bd59c3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:24.000118 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:23.994384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.29.145.3:46813)
0811 11:13:23.994576 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:23.994582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:23.994742 (+ 160us) server_negotiation.cc:407] Connection header received
0811 11:13:23.994785 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:23.994787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:23.994820 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:23.995147 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:23.995222 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.996328 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.998384 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:23.999432 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:23.999926 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:23.999998 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:24.000030 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 11:13:24.000062 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260811 11:13:24.000036 15592 tablet_service.cc:1467] Processing CreateTablet for tablet 30558408de574c86a64b6f50bd59c3aa (DEFAULT_TABLE table=test-table [id=8ec15c488a0145d3af26b512932122dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:24.000344 15592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30558408de574c86a64b6f50bd59c3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:24.000656 15897 tablet_service.cc:1467] Processing CreateTablet for tablet 30558408de574c86a64b6f50bd59c3aa (DEFAULT_TABLE table=test-table [id=8ec15c488a0145d3af26b512932122dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:24.000888 15897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30558408de574c86a64b6f50bd59c3aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:24.003443 15996 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Bootstrap starting.
I20260811 11:13:24.003875 15997 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Bootstrap starting.
I20260811 11:13:24.004108 15996 tablet_bootstrap.cc:654] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:24.004462 15997 tablet_bootstrap.cc:654] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:24.004658 15996 log.cc:824] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:24.005007 15997 log.cc:824] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:24.005332 15998 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap starting.
I20260811 11:13:24.005451 15996 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: No bootstrap required, opened a new log
I20260811 11:13:24.005529 15996 ts_tablet_manager.cc:1397] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:13:24.005753 15997 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: No bootstrap required, opened a new log
I20260811 11:13:24.005847 15997 ts_tablet_manager.cc:1397] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:13:24.006130 15998 tablet_bootstrap.cc:654] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:24.006671 15998 log.cc:824] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:24.007356 15998 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: No bootstrap required, opened a new log
I20260811 11:13:24.007436 15998 ts_tablet_manager.cc:1397] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 11:13:24.008018 15996 raft_consensus.cc:348] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.008018 15997 raft_consensus.cc:348] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.008229 15996 raft_consensus.cc:374] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:24.008231 15997 raft_consensus.cc:374] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:24.008273 15997 raft_consensus.cc:729] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d94e62a094409e8e18f0548489e3a8, State: Initialized, Role: FOLLOWER
I20260811 11:13:24.008273 15996 raft_consensus.cc:729] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a88b46d3c944f590d2804a8fbae9cc, State: Initialized, Role: FOLLOWER
I20260811 11:13:24.008399 15996 consensus_queue.cc:260] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.008438 15997 consensus_queue.cc:260] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.008750 15996 ts_tablet_manager.cc:1428] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:13:24.008773 15977 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
I20260811 11:13:24.008916 15818 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
I20260811 11:13:24.009119 15997 ts_tablet_manager.cc:1428] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:13:24.009249 15998 raft_consensus.cc:348] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.009430 15998 raft_consensus.cc:374] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:24.009478 15998 raft_consensus.cc:729] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ad1c4f45244aabbf9f8607667afcc9, State: Initialized, Role: FOLLOWER
I20260811 11:13:24.009636 15998 consensus_queue.cc:260] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.009972 15998 ts_tablet_manager.cc:1428] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 11:13:24.010106 15666 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
W20260811 11:13:24.045763 15819 tablet.cc:2369] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:13:24.129170 15667 tablet.cc:2369] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:13:24.218436 15978 tablet.cc:2369] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:24.238739 16005 raft_consensus.cc:482] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:24.238890 16005 raft_consensus.cc:504] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.239383 16005 leader_election.cc:288] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813), 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239)
I20260811 11:13:24.242729 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.239609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36239 (local address 127.29.145.1:51663)
0811 11:13:24.239757 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:24.239770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:24.239837 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:24.240082 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:24.240083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:24.240085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:24.240219 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:24.240221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.240957 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.240958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:24.241760 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:24.241761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.242451 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.242453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:24.242528 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:24.242532 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:13:24.242584 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 11:13:24.242619 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":12,"thread_start_us":77,"threads_started":1}
I20260811 11:13:24.242734 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.239710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:51663 (local address 127.29.145.2:36239)
0811 11:13:24.239762 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:24.239764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:24.239828 (+ 64us) server_negotiation.cc:407] Connection header received
0811 11:13:24.239981 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:24.239983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:24.239997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:24.240063 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:24.240298 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.240842 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.241889 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.242361 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.242425 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:24.242432 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:24.242607 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 11:13:24.242655 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:13:24.242816 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.239906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:47615 (local address 127.29.145.3:46813)
0811 11:13:24.239956 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:24.239958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:24.239984 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:13:24.240075 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:24.240076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:24.240086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:24.240146 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:24.240367 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.240839 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.242003 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.242386 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.242483 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:24.242491 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:24.242702 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 11:13:24.242746 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:13:24.242897 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.239685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:46813 (local address 127.29.145.1:47615)
0811 11:13:24.239886 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:24.239898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:24.240001 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:24.240175 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:24.240176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:24.240178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:24.240276 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:24.240278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.240936 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.240938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:24.241888 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:24.241892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.242506 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.242507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:24.242607 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:24.242612 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:13:24.242682 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 11:13:24.242724 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260811 11:13:24.243345 15766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30558408de574c86a64b6f50bd59c3aa" candidate_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d94e62a094409e8e18f0548489e3a8" is_pre_election: true
I20260811 11:13:24.243520 15766 raft_consensus.cc:2454] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ad1c4f45244aabbf9f8607667afcc9 in term 0.
I20260811 11:13:24.243517 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30558408de574c86a64b6f50bd59c3aa" candidate_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" is_pre_election: true
I20260811 11:13:24.243709 15920 raft_consensus.cc:2454] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ad1c4f45244aabbf9f8607667afcc9 in term 0.
I20260811 11:13:24.243813 15540 leader_election.cc:302] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d94e62a094409e8e18f0548489e3a8, f7ad1c4f45244aabbf9f8607667afcc9; no voters:
I20260811 11:13:24.244001 16005 raft_consensus.cc:2781] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:13:24.244056 16005 raft_consensus.cc:482] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:13:24.244102 16005 raft_consensus.cc:3037] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:24.244904 16005 raft_consensus.cc:504] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.245131 16005 leader_election.cc:288] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 1 election: Requested vote from peers e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813), 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239)
I20260811 11:13:24.245249 15766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30558408de574c86a64b6f50bd59c3aa" candidate_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d94e62a094409e8e18f0548489e3a8"
I20260811 11:13:24.245296 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30558408de574c86a64b6f50bd59c3aa" candidate_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
I20260811 11:13:24.245348 15766 raft_consensus.cc:3037] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:24.245366 15920 raft_consensus.cc:3037] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:24.246335 15920 raft_consensus.cc:2454] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ad1c4f45244aabbf9f8607667afcc9 in term 1.
I20260811 11:13:24.246335 15766 raft_consensus.cc:2454] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ad1c4f45244aabbf9f8607667afcc9 in term 1.
I20260811 11:13:24.246536 15540 leader_election.cc:302] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d94e62a094409e8e18f0548489e3a8, f7ad1c4f45244aabbf9f8607667afcc9; no voters:
I20260811 11:13:24.246647 16005 raft_consensus.cc:2781] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:24.246855 16005 raft_consensus.cc:686] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 LEADER]: Becoming Leader. State: Replica: f7ad1c4f45244aabbf9f8607667afcc9, State: Running, Role: LEADER
I20260811 11:13:24.246991 16005 consensus_queue.cc:237] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:24.247911 15452 catalog_manager.cc:5495] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1). New cstate: current_term: 1 leader_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } health_report { overall_health: UNKNOWN } } }
I20260811 11:13:24.283278 15993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.280388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.29.145.1:44569)
0811 11:13:24.280514 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:24.280516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:24.280698 (+ 182us) server_negotiation.cc:407] Connection header received
0811 11:13:24.280765 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:24.280767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:24.280790 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:24.280866 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:24.280978 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.281511 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.282474 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:24.282657 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:24.282747 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:24.282929 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:24.283037 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:13:24.283103 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:24.283193 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 11:13:24.283210 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:13:24.285157 15920 raft_consensus.cc:1261] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Refusing update from remote peer f7ad1c4f45244aabbf9f8607667afcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:24.285177 15766 raft_consensus.cc:1261] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Refusing update from remote peer f7ad1c4f45244aabbf9f8607667afcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:24.285531 16009 consensus_queue.cc:1035] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:24.285696 16005 consensus_queue.cc:1035] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:24.290885 15451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:24.290994 15451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:24.291834 15451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Sending DeleteTablet for 3 replicas of tablet 30558408de574c86a64b6f50bd59c3aa
I20260811 11:13:24.292157 15897 tablet_service.cc:1514] Processing DeleteTablet for tablet 30558408de574c86a64b6f50bd59c3aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:58452
I20260811 11:13:24.292214 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15523
I20260811 11:13:24.292354 15592 tablet_service.cc:1514] Processing DeleteTablet for tablet 30558408de574c86a64b6f50bd59c3aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:49674
I20260811 11:13:24.292402 15742 tablet_service.cc:1514] Processing DeleteTablet for tablet 30558408de574c86a64b6f50bd59c3aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:42152
I20260811 11:13:24.295485 16023 tablet_replica.cc:330] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: stopping tablet replica
I20260811 11:13:24.295605 16023 raft_consensus.cc:2229] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:24.295719 16023 raft_consensus.cc:2258] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:24.296170 16023 ts_tablet_manager.cc:1905] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:24.298048 16022 tablet_replica.cc:330] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: stopping tablet replica
I20260811 11:13:24.298197 16022 raft_consensus.cc:2229] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:24.298339 16022 raft_consensus.cc:2258] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:24.298326 16023 ts_tablet_manager.cc:1918] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:24.298400 16023 log.cc:1196] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/30558408de574c86a64b6f50bd59c3aa
I20260811 11:13:24.298660 16023 ts_tablet_manager.cc:1939] T 30558408de574c86a64b6f50bd59c3aa P 31d94e62a094409e8e18f0548489e3a8: Deleting consensus metadata
I20260811 11:13:24.298774 16022 ts_tablet_manager.cc:1905] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:24.299265 15436 catalog_manager.cc:4841] TS 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): tablet 30558408de574c86a64b6f50bd59c3aa (table test-table [id=8ec15c488a0145d3af26b512932122dd]) successfully deleted
I20260811 11:13:24.300348 16022 ts_tablet_manager.cc:1918] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:24.300429 16022 log.cc:1196] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/30558408de574c86a64b6f50bd59c3aa
I20260811 11:13:24.300683 16022 ts_tablet_manager.cc:1939] T 30558408de574c86a64b6f50bd59c3aa P e0a88b46d3c944f590d2804a8fbae9cc: Deleting consensus metadata
I20260811 11:13:24.301083 15437 catalog_manager.cc:4841] TS e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): tablet 30558408de574c86a64b6f50bd59c3aa (table test-table [id=8ec15c488a0145d3af26b512932122dd]) successfully deleted
W20260811 11:13:24.301954 15438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:44569 (error 108)
I20260811 11:13:24.302172 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15671
I20260811 11:13:24.303332 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:24.303191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44569 (local address 127.0.0.1:49694)
0811 11:13:24.303242 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:24.303291 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:13:24.303452 15438 catalog_manager.cc:4584] TS f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30558408de574c86a64b6f50bd59c3aa: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44569: connect: Connection refused (error 111)
I20260811 11:13:24.308799 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15824
I20260811 11:13:24.316609 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 15419
I20260811 11:13:44.325464 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--webserver_interface=127.29.145.62
--webserver_port=41869
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:13:44.425683 16028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:44.426030 16028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:44.426059 16028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:44.427304 16028 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:13:44.427346 16028 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:13:44.427361 16028 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:13:44.427371 16028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:13:44.427385 16028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:44.427398 16028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:44.427451 16028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:13:44.427469 16028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:13:44.429052 16028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:44.429365 16028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:44.429664 16028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:44.441445 16035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.441572 16036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.441718 16028 server_base.cc:1034] running on GCE node
W20260811 11:13:44.441572 16038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.442158 16028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:44.442571 16028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:44.443744 16028 hybrid_clock.cc:648] HybridClock initialized: now 1786446824443707 us; error 53 us; skew 500 ppm
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:44.447129 16028 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:13:44.447753 16028 webserver.cc:460] Webserver started at http://127.29.145.62:41869/ using document root <none> and password file <none>
I20260811 11:13:44.448072 16028 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:44.448177 16028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:44.449915 16028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 11:13:44.450793 16046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.451078 16028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:44.451207 16028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "95f02978e6e94ba88869a4459808f9af"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "c51ad1ae9f40d699c40d8329ba500f20"
server_key_iv: "a1b167597ab9debf5ba909520607f742"
server_key_version: "encryptionkey@0"
I20260811 11:13:44.451794 16028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:44.464784 16028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:44.467638 16028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:44.467833 16028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:44.473575 16028 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37699
I20260811 11:13:44.473634 16104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37699 every 8 connection(s)
I20260811 11:13:44.474202 16028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:13:44.476178 16105 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:13:44.477000 16105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap starting.
I20260811 11:13:44.484127 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16028
I20260811 11:13:44.484520 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:44569
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=38005
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:44.489221 16105 log.cc:824] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:44.492865 16105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:44.493343 16105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap complete.
I20260811 11:13:44.498266 16105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:44.498775 16105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Initialized, Role: FOLLOWER
I20260811 11:13:44.499027 16105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:44.499136 16105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:13:44.499209 16105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:13:44.499387 16105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:13:44.501376 16105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:44.501523 16105 leader_election.cc:302] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f02978e6e94ba88869a4459808f9af; no voters:
I20260811 11:13:44.501741 16105 leader_election.cc:288] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:13:44.501859 16110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:13:44.502103 16105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:13:44.502148 16110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 LEADER]: Becoming Leader. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Running, Role: LEADER
I20260811 11:13:44.502463 16110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:13:44.502949 16112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f02978e6e94ba88869a4459808f9af. Latest consensus state: current_term: 2 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:13:44.503058 16112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:44.503526 16111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:13:44.503753 16120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:13:44.503784 16111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
I20260811 11:13:44.504935 16120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:13:44.505053 16120 catalog_manager.cc:1259] Loaded cluster ID: 65fbfe6fcc6c47e4a03e1cd46b0835af
I20260811 11:13:44.505089 16120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:13:44.506392 16120 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:13:44.506620 16120 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Loaded TSK: 0
I20260811 11:13:44.516849 16120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Generated new TSK 1
I20260811 11:13:44.517004 16120 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:13:44.586426 16108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:44.586707 16108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:44.586745 16108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:44.587983 16108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:44.588052 16108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:44.588100 16108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:13:44.589673 16108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:44569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=38005
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:44.590005 16108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:44.590291 16108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:44.596046 16136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.596089 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.596062 16137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.596320 16108 server_base.cc:1034] running on GCE node
I20260811 11:13:44.596654 16108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:44.596951 16108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:44.598122 16108 hybrid_clock.cc:648] HybridClock initialized: now 1786446824598096 us; error 45 us; skew 500 ppm
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:44.601256 16108 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:13:44.601915 16108 webserver.cc:460] Webserver started at http://127.29.145.1:38005/ using document root <none> and password file <none>
I20260811 11:13:44.602258 16108 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:44.602358 16108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:44.604247 16108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:44.605058 16148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.605297 16108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:13:44.605412 16108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "606d3394d0c8264c13f9a1bd33094267"
server_key_iv: "c2d07b7c15a772e46abd151d1c28ce1d"
server_key_version: "encryptionkey@0"
I20260811 11:13:44.606029 16108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:44.633620 16108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:44.636456 16108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:44.636623 16108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:44.636931 16108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:44.637554 16157 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:13:44.639580 16108 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:13:44.639647 16108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 11:13:44.639688 16108 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:13:44.640741 16108 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:13:44.640790 16108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:44.640908 16157 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap starting.
I20260811 11:13:44.649398 16108 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44569
I20260811 11:13:44.649675 16280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44569 every 8 connection(s)
I20260811 11:13:44.650015 16108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:13:44.653573 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16108
I20260811 11:13:44.653970 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:36239
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=39839
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:13:44.654284 16157 log.cc:824] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:44.657196 16157 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:13:44.657526 16157 tablet_bootstrap.cc:492] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap complete.
I20260811 11:13:44.658620 16157 ts_tablet_manager.cc:1397] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.010s
I20260811 11:13:44.660064 16157 raft_consensus.cc:348] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:44.660373 16157 raft_consensus.cc:729] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ad1c4f45244aabbf9f8607667afcc9, State: Initialized, Role: FOLLOWER
I20260811 11:13:44.660584 16157 consensus_queue.cc:260] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } } peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } }
I20260811 11:13:44.660944 16157 ts_tablet_manager.cc:1428] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:44.673347 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.651087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:55857 (local address 127.29.145.62:37699)
0811 11:13:44.651382 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.651394 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.661008 (+ 9614us) server_negotiation.cc:407] Connection header received
0811 11:13:44.661280 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.661286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.661337 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.661477 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.662607 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.663651 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.665397 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.665991 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.666097 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.669146 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.669157 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.669157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.669173 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.671255 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.671779 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.671788 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.671788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.671909 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.672213 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.672214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.672214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.672501 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.672611 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.672950 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.673104 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":162,"threads_started":1}
I20260811 11:13:44.673347 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.650955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:55857)
0811 11:13:44.654871 (+ 3916us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.660796 (+ 5925us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.661058 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.661548 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.661553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.662122 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:44.662446 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.662454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.663787 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.663788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.665282 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.665287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.666096 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.666098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.666271 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.666961 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:44.666974 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:44.668996 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:44.671366 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.671376 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.671379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.671647 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.672005 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.672006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.672007 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.672134 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.672599 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:44.672607 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:44.672761 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.672884 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.673104 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3859,"thread_start_us":100,"threads_started":1}
I20260811 11:13:44.674794 16281 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:44.674926 16281 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:44.675154 16281 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:44.676092 16065 ts_manager.cc:194] Registered new tserver with Master: f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569)
I20260811 11:13:44.677537 16065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:55857
I20260811 11:13:44.678725 16281 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.1@KRBTEST.COM
I20260811 11:13:44.687186 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.676974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44569 (local address 127.0.0.1:50700)
0811 11:13:44.677290 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.677322 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.677431 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.677801 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.677807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.678267 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:44.678346 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.678358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.679693 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.679694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.680388 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.680391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.681032 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.681033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.681106 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.681746 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:44.681752 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:44.683400 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:44.685431 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.685438 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.685441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.686061 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.686367 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.686368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.686368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.686475 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.686832 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:44.686839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:44.686911 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.686985 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.687105 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":153,"threads_started":1}
I20260811 11:13:44.687261 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.677043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.29.145.1:44569)
0811 11:13:44.677562 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.677566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.677575 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:44.677629 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.677631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.677650 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.677724 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.678447 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.679571 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.680478 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.680936 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.681010 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.683488 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.683498 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.683499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.683507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.685338 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.686194 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.686198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.686198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.686268 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.686551 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.686552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.686553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.686738 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.686822 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.687054 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.687159 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":100,"threads_started":1}
I20260811 11:13:44.687868 16185 tablet_service.cc:1514] Processing DeleteTablet for tablet 30558408de574c86a64b6f50bd59c3aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:24 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:50700
I20260811 11:13:44.688616 16293 tablet_replica.cc:330] stopping tablet replica
I20260811 11:13:44.688742 16293 raft_consensus.cc:2229] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:44.688848 16293 raft_consensus.cc:2258] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:44.689217 16293 ts_tablet_manager.cc:1905] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:44.691597 16293 ts_tablet_manager.cc:1918] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:44.691671 16293 log.cc:1196] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal/wals/30558408de574c86a64b6f50bd59c3aa
I20260811 11:13:44.691929 16293 ts_tablet_manager.cc:1939] T 30558408de574c86a64b6f50bd59c3aa P f7ad1c4f45244aabbf9f8607667afcc9: Deleting consensus metadata
I20260811 11:13:44.692566 16049 catalog_manager.cc:4841] TS f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569): tablet 30558408de574c86a64b6f50bd59c3aa (table test-table [id=8ec15c488a0145d3af26b512932122dd]) successfully deleted
W20260811 11:13:44.758369 16287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:44.758666 16287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:44.758708 16287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:44.759949 16287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:44.760030 16287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:44.760077 16287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:13:44.761657 16287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:36239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=39839
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:44.761993 16287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:44.762264 16287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:44.768525 16303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.768548 16301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.768718 16287 server_base.cc:1034] running on GCE node
W20260811 11:13:44.768534 16300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.769217 16287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:44.769580 16287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:44.770768 16287 hybrid_clock.cc:648] HybridClock initialized: now 1786446824770741 us; error 45 us; skew 500 ppm
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:44.774253 16287 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:13:44.774933 16287 webserver.cc:460] Webserver started at http://127.29.145.2:39839/ using document root <none> and password file <none>
I20260811 11:13:44.775261 16287 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:44.775372 16287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:44.777248 16287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:13:44.778122 16314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.778321 16287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:44.778441 16287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "31d94e62a094409e8e18f0548489e3a8"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "02f4d32592f810078d1ac126bb27b518"
server_key_iv: "e4f885ca95b08ce5d747b700a4bee7c5"
server_key_version: "encryptionkey@0"
I20260811 11:13:44.778899 16287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:44.789604 16287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:44.792100 16287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:44.792302 16287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:44.792618 16287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:44.793001 16287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:44.793056 16287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.793102 16287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:44.793129 16287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.800369 16287 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36239
I20260811 11:13:44.800433 16442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36239 every 8 connection(s)
I20260811 11:13:44.800863 16287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:13:44.805472 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16287
I20260811 11:13:44.805804 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:46813
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=45269
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:44.813272 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.801881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:57979 (local address 127.29.145.62:37699)
0811 11:13:44.801967 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.801970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.802264 (+ 294us) server_negotiation.cc:407] Connection header received
0811 11:13:44.803086 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.803088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.803105 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.803161 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.804571 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.805157 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.806553 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.806940 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.807023 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.809695 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.809697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.809698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.809708 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.811525 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.812046 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.812048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.812049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.812124 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.812423 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.812424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.812424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.812565 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.812698 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.813037 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.813159 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:13:44.813486 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.801875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:57979)
0811 11:13:44.802089 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.802134 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.802976 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.803241 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.803245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.804185 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:44.804442 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.804448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.805276 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.805277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.806433 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.806436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.807039 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.807040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.807168 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.807884 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:44.807894 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:44.809575 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:44.811620 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.811629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.811632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.811954 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.812209 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.812210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.812211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.812339 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.812683 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:44.812691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:44.812871 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.812997 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.813195 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260811 11:13:44.814270 16443 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:44.814415 16443 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:44.814670 16443 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:44.815163 16065 ts_manager.cc:194] Registered new tserver with Master: 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239)
I20260811 11:13:44.815804 16065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:57979
W20260811 11:13:44.903568 16448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:13:44.903961 16448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:13:44.903996 16448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:13:44.905412 16448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:13:44.905465 16448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:13:44.905498 16448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:13:44.907130 16448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:46813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=45269
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:13:44.907379 16448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:13:44.907620 16448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:13:44.913694 16455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.913688 16456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:13:44.913683 16458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:13:44.914021 16448 server_base.cc:1034] running on GCE node
I20260811 11:13:44.914269 16448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:13:44.914584 16448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:13:44.915747 16448 hybrid_clock.cc:648] HybridClock initialized: now 1786446824915724 us; error 39 us; skew 500 ppm
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:13:44.918905 16448 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:13:44.919497 16448 webserver.cc:460] Webserver started at http://127.29.145.3:45269/ using document root <none> and password file <none>
I20260811 11:13:44.919780 16448 fs_manager.cc:359] Metadata directory not provided
I20260811 11:13:44.919888 16448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:13:44.921658 16448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:13:44.922621 16465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.922861 16448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:13:44.923000 16448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "1c0c9f1ebb919a0bdf0ea34ca1623f91"
server_key_iv: "197ea9ab9f1f06d89a2587fd16dd0fc2"
server_key_version: "encryptionkey@0"
I20260811 11:13:44.923478 16448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:13:44.938745 16448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:13:44.941555 16448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:13:44.941774 16448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:13:44.942157 16448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:13:44.942516 16448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:13:44.942552 16448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.942579 16448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:13:44.942591 16448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:13:44.949601 16448 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:46813
I20260811 11:13:44.949738 16593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:46813 every 8 connection(s)
I20260811 11:13:44.950143 16448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:13:44.954375 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16448
Aug 11 11:13:44 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446824, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:13:44.962651 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.951062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:39923 (local address 127.29.145.62:37699)
0811 11:13:44.951171 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.951174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.951388 (+ 214us) server_negotiation.cc:407] Connection header received
0811 11:13:44.952019 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.952021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.952033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.952084 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.953248 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.953805 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.955653 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.956002 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.956103 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.959027 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.959029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.959029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.959037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.960751 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.961379 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.961380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.961380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.961439 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.961792 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.961793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.961794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.961985 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.962063 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.962428 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.962559 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:13:44.962941 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.951025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:39923)
0811 11:13:44.951235 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.951280 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.951918 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.952161 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.952165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.952890 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:13:44.953149 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.953154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.953938 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.953939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.955558 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.955561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.956130 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.956131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.956309 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.956976 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:13:44.956989 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:13:44.958924 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:13:44.960852 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.960862 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.960866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.961279 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.961541 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:13:44.961542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:13:44.961543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:13:44.961711 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:13:44.962062 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:13:44.962068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:13:44.962257 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.962393 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.962629 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260811 11:13:44.963372 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.954886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.29.145.62:37699)
0811 11:13:44.955257 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.955260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.955268 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:13:44.955292 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.955293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.955307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.955391 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.956255 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.957006 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.957797 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.958170 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.958257 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.959761 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.959763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.959763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.959770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.961794 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.962219 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.962220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.962220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.962290 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.962534 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.962535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.962536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.962682 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.962893 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.963141 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.963244 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":75,"threads_started":1}
I20260811 11:13:44.963589 16594 heartbeater.cc:347] Connected to a master server at 127.29.145.62:37699
I20260811 11:13:44.963737 16594 heartbeater.cc:464] Registering TS with master...
I20260811 11:13:44.964048 16594 heartbeater.cc:511] Master 127.29.145.62:37699 requested a full tablet report, sending...
I20260811 11:13:44.964535 16065 ts_manager.cc:194] Registered new tserver with Master: e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
I20260811 11:13:44.964974 16065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:39923
I20260811 11:13:44.975229 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:13:44.979086 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.975740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36570 (local address 127.29.145.62:37699)
0811 11:13:44.975838 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.975842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.975974 (+ 132us) server_negotiation.cc:407] Connection header received
0811 11:13:44.976111 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.976112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.976176 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.976273 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:44.976493 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.977089 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.978094 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.978347 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.978454 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.978674 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:44.978750 (+ 76us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 11:13:44.979001 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36570: token expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 11:13:44.979179 16598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36570: token expired
W20260811 11:13:44.980096 15981 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.29.145.62:37699)
I20260811 11:13:44.987450 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.980448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.29.145.62:37699)
0811 11:13:44.980494 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.980496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.980505 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:44.980556 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.980558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.980568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.980623 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:13:44.981227 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.981786 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.982746 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.982978 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.983073 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.984446 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:13:44.984448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:13:44.984448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:13:44.984457 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:13:44.985975 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.986509 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.986510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.986511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.986598 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:13:44.986834 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:13:44.986835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:13:44.986836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:13:44.986984 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:13:44.987111 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.987205 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.987308 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:13:44.988987 16065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:13:44.991173 16065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:13:44.994104 16185 tablet_service.cc:1467] Processing CreateTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f (DEFAULT_TABLE table=test-table [id=fa38becc93c847a79e78bc78187df0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:44.994379 16185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727839690df6403c9d0e1bdb5cd4b60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:44.996101 16157 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap starting.
I20260811 11:13:44.996544 16157 tablet_bootstrap.cc:654] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:44.997641 16157 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: No bootstrap required, opened a new log
I20260811 11:13:44.997702 16157 ts_tablet_manager.cc:1397] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:13:44.997905 16157 raft_consensus.cc:348] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:44.998000 16157 raft_consensus.cc:374] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:44.998085 16157 raft_consensus.cc:729] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ad1c4f45244aabbf9f8607667afcc9, State: Initialized, Role: FOLLOWER
I20260811 11:13:44.998152 16157 consensus_queue.cc:260] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:44.998270 16157 ts_tablet_manager.cc:1428] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:13:44.998329 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.993595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:46813 (local address 127.0.0.1:37540)
0811 11:13:44.993741 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.993753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.993832 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.994266 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.994268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.994271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:44.994323 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.994325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.995786 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.995787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.996972 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.996975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.998018 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.998020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.998107 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.998128 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.998186 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.998241 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260811 11:13:44.998893 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.993762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.29.145.3:46813)
0811 11:13:44.994078 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.994082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.994091 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:44.994133 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.994136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.994153 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.994402 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:44.994510 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.995681 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.997062 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.997906 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.998608 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.998704 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.998736 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.998798 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":156,"threads_started":1}
I20260811 11:13:44.999380 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.993512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36239 (local address 127.0.0.1:47390)
0811 11:13:44.993640 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:44.993654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:44.993705 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:44.994236 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:44.994237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:44.994240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:44.994305 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.994306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.997054 (+ 2748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.997055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.997977 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:44.997980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.999088 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.999089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:44.999174 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.999178 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:13:44.999249 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:13:44.999310 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 11:13:44.999647 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:44.993743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.29.145.2:36239)
0811 11:13:44.994003 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:44.994010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:44.994023 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:13:44.994080 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:44.994083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:44.994105 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:44.994215 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:44.994373 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.995693 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.998103 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:44.998905 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:44.998991 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:44.999053 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:44.999508 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 11:13:44.999562 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":152,"threads_started":1}
I20260811 11:13:44.999858 16521 tablet_service.cc:1467] Processing CreateTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f (DEFAULT_TABLE table=test-table [id=fa38becc93c847a79e78bc78187df0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:45.000330 16521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727839690df6403c9d0e1bdb5cd4b60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:45.000538 16368 tablet_service.cc:1467] Processing CreateTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f (DEFAULT_TABLE table=test-table [id=fa38becc93c847a79e78bc78187df0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:13:45.000982 16368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 727839690df6403c9d0e1bdb5cd4b60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:13:45.004640 16604 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Bootstrap starting.
I20260811 11:13:45.005203 16605 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Bootstrap starting.
I20260811 11:13:45.005563 16604 tablet_bootstrap.cc:654] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:45.006132 16605 tablet_bootstrap.cc:654] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Neither blocks nor log segments found. Creating new log.
I20260811 11:13:45.006448 16604 log.cc:824] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:45.006880 16605 log.cc:824] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:13:45.007468 16604 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: No bootstrap required, opened a new log
I20260811 11:13:45.007607 16604 ts_tablet_manager.cc:1397] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:13:45.007764 16605 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: No bootstrap required, opened a new log
I20260811 11:13:45.007887 16605 ts_tablet_manager.cc:1397] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:13:45.009358 16604 raft_consensus.cc:348] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.009563 16604 raft_consensus.cc:374] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:45.009614 16604 raft_consensus.cc:729] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a88b46d3c944f590d2804a8fbae9cc, State: Initialized, Role: FOLLOWER
I20260811 11:13:45.009543 16605 raft_consensus.cc:348] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.009758 16605 raft_consensus.cc:374] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:13:45.009761 16604 consensus_queue.cc:260] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.009876 16605 raft_consensus.cc:729] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d94e62a094409e8e18f0548489e3a8, State: Initialized, Role: FOLLOWER
I20260811 11:13:45.010025 16605 consensus_queue.cc:260] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.010186 16604 ts_tablet_manager.cc:1428] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:13:45.010216 16594 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
I20260811 11:13:45.010497 16443 heartbeater.cc:503] Master 127.29.145.62:37699 was elected leader, sending a full tablet report...
I20260811 11:13:45.010591 16605 ts_tablet_manager.cc:1428] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260811 11:13:45.051682 16445 tablet.cc:2369] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:13:45.155316 16283 tablet.cc:2369] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:13:45.200906 16595 tablet.cc:2369] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:13:45.248958 16608 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:13:45.249150 16608 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.249750 16608 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569)
I20260811 11:13:45.253428 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.250202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:46583 (local address 127.29.145.2:36239)
0811 11:13:45.250273 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:45.250275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:45.250453 (+ 178us) server_negotiation.cc:407] Connection header received
0811 11:13:45.250536 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:45.250550 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:45.250560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:45.250645 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:45.250849 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.251429 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.252576 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.253008 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.253106 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:45.253119 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:45.253281 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:13:45.253327 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:13:45.253468 16596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.250078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44569 (local address 127.29.145.3:59595)
0811 11:13:45.250163 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:45.250183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:45.250280 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:45.250691 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:45.250693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:45.250694 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:45.250788 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:45.250790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.251554 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.251555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:45.252467 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:45.252470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.253158 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.253160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:45.253239 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:45.253243 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:13:45.253299 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 11:13:45.253365 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":18}
I20260811 11:13:45.253486 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.250285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59595 (local address 127.29.145.1:44569)
0811 11:13:45.250468 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:45.250470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:45.250479 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:45.250514 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:45.250515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:45.250547 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:45.250678 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:13:45.250908 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.251455 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.252574 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.253065 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.253164 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:45.253260 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:45.253317 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 11:13:45.253373 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260811 11:13:45.253496 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.250109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36239 (local address 127.29.145.3:46583)
0811 11:13:45.250345 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:45.250356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:13:45.250463 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:13:45.250650 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:13:45.250652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:13:45.250654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:13:45.250767 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:45.250769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.251524 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.251526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:45.252470 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:13:45.252473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.253104 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.253105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:13:45.253186 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:45.253191 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:13:45.253273 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:13:45.253318 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":49,"thread_start_us":153,"threads_started":1}
I20260811 11:13:45.253873 16222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727839690df6403c9d0e1bdb5cd4b60f" candidate_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" is_pre_election: true
I20260811 11:13:45.253974 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727839690df6403c9d0e1bdb5cd4b60f" candidate_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d94e62a094409e8e18f0548489e3a8" is_pre_election: true
I20260811 11:13:45.254098 16222 raft_consensus.cc:2454] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a88b46d3c944f590d2804a8fbae9cc in term 0.
I20260811 11:13:45.254184 16393 raft_consensus.cc:2454] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a88b46d3c944f590d2804a8fbae9cc in term 0.
I20260811 11:13:45.254407 16469 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a88b46d3c944f590d2804a8fbae9cc, f7ad1c4f45244aabbf9f8607667afcc9; no voters:
I20260811 11:13:45.254591 16608 raft_consensus.cc:2781] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:13:45.254676 16608 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:13:45.254719 16608 raft_consensus.cc:3037] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:45.255946 16608 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.256258 16608 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [CANDIDATE]: Term 1 election: Requested vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569)
I20260811 11:13:45.256637 16393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727839690df6403c9d0e1bdb5cd4b60f" candidate_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d94e62a094409e8e18f0548489e3a8"
I20260811 11:13:45.256654 16222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "727839690df6403c9d0e1bdb5cd4b60f" candidate_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
I20260811 11:13:45.256758 16393 raft_consensus.cc:3037] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:45.256858 16222 raft_consensus.cc:3037] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:13:45.257709 16393 raft_consensus.cc:2454] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a88b46d3c944f590d2804a8fbae9cc in term 1.
I20260811 11:13:45.257718 16222 raft_consensus.cc:2454] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a88b46d3c944f590d2804a8fbae9cc in term 1.
I20260811 11:13:45.257932 16467 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc; no voters:
I20260811 11:13:45.258055 16608 raft_consensus.cc:2781] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:13:45.258244 16608 raft_consensus.cc:686] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 1 LEADER]: Becoming Leader. State: Replica: e0a88b46d3c944f590d2804a8fbae9cc, State: Running, Role: LEADER
I20260811 11:13:45.258369 16608 consensus_queue.cc:237] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:13:45.259490 16065 catalog_manager.cc:5495] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3). New cstate: current_term: 1 leader_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } health_report { overall_health: UNKNOWN } } }
I20260811 11:13:45.286607 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.284189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37550 (local address 127.29.145.3:46813)
0811 11:13:45.284284 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 11:13:45.284285 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:13:45.284294 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:13:45.284333 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:13:45.284333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:13:45.284351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:13:45.284416 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:13:45.284532 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.285007 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.285795 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:13:45.286114 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:13:45.286178 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:13:45.286307 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:13:45.286417 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:13:45.286474 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:13:45.286508 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 11:13:45.286530 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:13:45.288637 16393 raft_consensus.cc:1261] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Refusing update from remote peer e0a88b46d3c944f590d2804a8fbae9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:45.288630 16222 raft_consensus.cc:1261] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Refusing update from remote peer e0a88b46d3c944f590d2804a8fbae9cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:13:45.289005 16608 consensus_queue.cc:1035] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:45.289202 16612 consensus_queue.cc:1035] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:13:45.291538 16615 mvcc.cc:205] Tried to move back new op lower bound from 7317286196379254784 to 7317286196259123200. Current Snapshot: MvccSnapshot[applied={T|T < 7317286196379254784}]
I20260811 11:13:45.291873 16618 mvcc.cc:205] Tried to move back new op lower bound from 7317286196379254784 to 7317286196259123200. Current Snapshot: MvccSnapshot[applied={T|T < 7317286196379254784}]
I20260811 11:13:45.291934 16616 mvcc.cc:205] Tried to move back new op lower bound from 7317286196379254784 to 7317286196259123200. Current Snapshot: MvccSnapshot[applied={T|T < 7317286196379254784}]
I20260811 11:13:45.294931 16065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:45.295060 16065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:13:45.295877 16065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Sending DeleteTablet for 3 replicas of tablet 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:13:45.296260 16185 tablet_service.cc:1514] Processing DeleteTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:45 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:50700
I20260811 11:13:45.296255 16521 tablet_service.cc:1514] Processing DeleteTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:45 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:37540
I20260811 11:13:45.296288 16368 tablet_service.cc:1514] Processing DeleteTablet for tablet 727839690df6403c9d0e1bdb5cd4b60f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:13:45 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:47390
I20260811 11:13:45.296517 16628 tablet_replica.cc:330] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: stopping tablet replica
I20260811 11:13:45.296586 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16108
I20260811 11:13:45.296633 16628 raft_consensus.cc:2229] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:13:45.296780 16628 raft_consensus.cc:2258] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:45.304358 16628 ts_tablet_manager.cc:1905] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:45.306756 16628 ts_tablet_manager.cc:1918] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:45.306828 16628 log.cc:1196] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:13:45.307117 16628 ts_tablet_manager.cc:1939] T 727839690df6403c9d0e1bdb5cd4b60f P e0a88b46d3c944f590d2804a8fbae9cc: Deleting consensus metadata
I20260811 11:13:45.307533 16630 tablet_replica.cc:330] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: stopping tablet replica
I20260811 11:13:45.307624 16048 catalog_manager.cc:4841] TS e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): tablet 727839690df6403c9d0e1bdb5cd4b60f (table test-table [id=fa38becc93c847a79e78bc78187df0f8]) successfully deleted
I20260811 11:13:45.307695 16630 raft_consensus.cc:2229] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:13:45.307837 16630 raft_consensus.cc:2258] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:13:45.308388 16630 ts_tablet_manager.cc:1905] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:13:45.310406 16630 ts_tablet_manager.cc:1918] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:13:45.310488 16630 log.cc:1196] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:13:45.310760 16630 ts_tablet_manager.cc:1939] T 727839690df6403c9d0e1bdb5cd4b60f P 31d94e62a094409e8e18f0548489e3a8: Deleting consensus metadata
I20260811 11:13:45.311236 16047 catalog_manager.cc:4841] TS 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): tablet 727839690df6403c9d0e1bdb5cd4b60f (table test-table [id=fa38becc93c847a79e78bc78187df0f8]) successfully deleted
W20260811 11:13:45.313575 16049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:44569 (error 108)
I20260811 11:13:45.313944 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16287
I20260811 11:13:45.314335 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:13:45.314213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:44569 (local address 127.0.0.1:50712)
0811 11:13:45.314269 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:13:45.314301 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 11:13:45.315394 16049 catalog_manager.cc:4584] TS f7ad1c4f45244aabbf9f8607667afcc9 (127.29.145.1:44569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 727839690df6403c9d0e1bdb5cd4b60f: Network error: Client connection negotiation failed: client connection to 127.29.145.1:44569: connect: Connection refused (error 111)
I20260811 11:13:45.322551 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16448
I20260811 11:13:45.330659 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16028
I20260811 11:14:05.339617 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--webserver_interface=127.29.145.62
--webserver_port=41869
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:05.445341 16634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:05.445619 16634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:05.445647 16634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:05.446969 16634 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:14:05.447014 16634 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:14:05.447028 16634 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:14:05.447041 16634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:05.447053 16634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:05.447105 16634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:05.447124 16634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:05.447146 16634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:05.448791 16634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:05.449113 16634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:05.449409 16634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:05.456501 16642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:05.456470 16634 server_base.cc:1034] running on GCE node
W20260811 11:14:05.456501 16639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.456530 16640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:05.457311 16634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:05.457696 16634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:05.458902 16634 hybrid_clock.cc:648] HybridClock initialized: now 1786446845458891 us; error 41 us; skew 500 ppm
Aug 11 11:14:05 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:05.462698 16634 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:05.463340 16634 webserver.cc:460] Webserver started at http://127.29.145.62:41869/ using document root <none> and password file <none>
I20260811 11:14:05.463680 16634 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:05.463737 16634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:05.465473 16634 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:14:05.466449 16651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.466686 16634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:05.466792 16634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "95f02978e6e94ba88869a4459808f9af"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "c51ad1ae9f40d699c40d8329ba500f20"
server_key_iv: "a1b167597ab9debf5ba909520607f742"
server_key_version: "encryptionkey@0"
I20260811 11:14:05.467238 16634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:05.496165 16634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:05.499171 16634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:05.499323 16634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:05.504684 16634 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37699
I20260811 11:14:05.504756 16713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37699 every 8 connection(s)
I20260811 11:14:05.505234 16634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:05.507198 16714 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:14:05.507946 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap starting.
I20260811 11:14:05.508782 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16634
I20260811 11:14:05.509161 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:44569
--local_ip_for_outbound_sockets=127.29.145.1
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=38005
--webserver_interface=127.29.145.1
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:14:05.520974 16714 log.cc:824] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:05.523950 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:14:05.524367 16714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Bootstrap complete.
I20260811 11:14:05.527251 16714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:14:05.527570 16714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Initialized, Role: FOLLOWER
I20260811 11:14:05.527793 16714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:14:05.527891 16714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:05.527972 16714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:05.528033 16714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 2 FOLLOWER]: Advancing to term 3
I20260811 11:14:05.529285 16714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:14:05.529433 16714 leader_election.cc:302] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f02978e6e94ba88869a4459808f9af; no voters:
I20260811 11:14:05.529695 16714 leader_election.cc:288] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 11:14:05.529747 16720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 3 FOLLOWER]: Leader election won for term 3
I20260811 11:14:05.529995 16720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [term 3 LEADER]: Becoming Leader. State: Replica: 95f02978e6e94ba88869a4459808f9af, State: Running, Role: LEADER
I20260811 11:14:05.530100 16714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:05.530217 16720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } }
I20260811 11:14:05.530666 16720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f02978e6e94ba88869a4459808f9af. Latest consensus state: current_term: 3 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:14:05.530745 16722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "95f02978e6e94ba88869a4459808f9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f02978e6e94ba88869a4459808f9af" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37699 } } }
I20260811 11:14:05.530884 16720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:05.530895 16722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:05.531190 16732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:05.532421 16732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:05.532577 16732 catalog_manager.cc:1259] Loaded cluster ID: 65fbfe6fcc6c47e4a03e1cd46b0835af
I20260811 11:14:05.532627 16732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:05.533535 16732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:05.533762 16732 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Loaded TSK: 0
I20260811 11:14:05.533824 16732 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 95f02978e6e94ba88869a4459808f9af: Loaded TSK: 1
F20260811 11:14:05.534104 16732 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f64889169ad google::LogMessage::Fail() at ??:0
@ 0x7f648891988a google::LogMessage::SendToLog() at ??:0
@ 0x7f64889164de google::LogMessage::Flush() at ??:0
@ 0x7f6488917a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f648b083104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f648b080d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f64908701cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f649087048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6490870594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6490836c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6490840ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f648a7e40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f648a7d9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f648b63d6db start_thread at ??:0
@ 0x7f648741461f clone at ??:0
W20260811 11:14:05.633404 16716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:05.633698 16716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:05.633729 16716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:05.635200 16716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:05.635267 16716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:05.635301 16716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:05.637199 16716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:44569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=38005
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:05.637624 16716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:05.638003 16716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:05.644793 16770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.644820 16767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.644920 16772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:05.645469 16716 server_base.cc:1034] running on GCE node
I20260811 11:14:05.645725 16716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:05.646045 16716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:05.647243 16716 hybrid_clock.cc:648] HybridClock initialized: now 1786446845647227 us; error 36 us; skew 500 ppm
Aug 11 11:14:05 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:05.650842 16716 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:14:05.651533 16716 webserver.cc:460] Webserver started at http://127.29.145.1:38005/ using document root <none> and password file <none>
I20260811 11:14:05.651883 16716 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:05.651981 16716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:05.654282 16716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:05.655436 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.655742 16716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:05.655858 16716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "f7ad1c4f45244aabbf9f8607667afcc9"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "606d3394d0c8264c13f9a1bd33094267"
server_key_iv: "c2d07b7c15a772e46abd151d1c28ce1d"
server_key_version: "encryptionkey@0"
I20260811 11:14:05.656424 16716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:05.692061 16716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:05.695350 16716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:05.695564 16716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:05.695901 16716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:05.696558 16788 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:14:05.698429 16716 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:14:05.698498 16716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 11:14:05.698536 16716 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:14:05.699592 16716 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:14:05.699643 16716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:05.699791 16788 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap starting.
I20260811 11:14:05.708770 16716 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:44569
I20260811 11:14:05.709110 16918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:44569 every 8 connection(s)
I20260811 11:14:05.709349 16716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:14:05.713078 16788 log.cc:824] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:05.715953 16788 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:14:05.716310 16788 tablet_bootstrap.cc:492] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Bootstrap complete.
I20260811 11:14:05.717594 16788 ts_tablet_manager.cc:1397] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.008s
I20260811 11:14:05.719184 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16716
I20260811 11:14:05.718987 16788 raft_consensus.cc:348] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:05.719427 16788 raft_consensus.cc:729] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ad1c4f45244aabbf9f8607667afcc9, State: Initialized, Role: FOLLOWER
I20260811 11:14:05.719552 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:36239
--local_ip_for_outbound_sockets=127.29.145.2
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=39839
--webserver_interface=127.29.145.2
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:14:05.719650 16788 consensus_queue.cc:260] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:05.720032 16788 ts_tablet_manager.cc:1428] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
W20260811 11:14:05.783339 16922 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:05.710252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:43147)
0811 11:14:05.710512 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.710575 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:05.711638 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:05.783063 (+ 71425us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":117,"threads_started":1}
W20260811 11:14:05.783712 16919 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:37699 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:37699: Client connection negotiation failed: client connection to 127.29.145.62:37699: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:14:05.784168 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.784004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:52001)
0811 11:14:05.784086 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.784138 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 11:14:05.784570 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.784471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:36707)
0811 11:14:05.784519 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.784544 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 11:14:05.784699 16919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:14:05.815835 16925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:05.816059 16925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:05.816109 16925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:05.817279 16925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:05.817350 16925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:05.817400 16925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:05.818948 16925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:36239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=39839
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:05.819217 16925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:05.819454 16925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:05.824553 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.824610 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.824570 16931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:05.824887 16925 server_base.cc:1034] running on GCE node
I20260811 11:14:05.825136 16925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:05.825397 16925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:05.826575 16925 hybrid_clock.cc:648] HybridClock initialized: now 1786446845826548 us; error 59 us; skew 500 ppm
Aug 11 11:14:05 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:05.829365 16925 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:14:05.829988 16925 webserver.cc:460] Webserver started at http://127.29.145.2:39839/ using document root <none> and password file <none>
I20260811 11:14:05.830284 16925 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:05.830363 16925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:05.831871 16925 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:14:05.832540 16942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.832710 16925 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:14:05.832825 16925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "31d94e62a094409e8e18f0548489e3a8"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "02f4d32592f810078d1ac126bb27b518"
server_key_iv: "e4f885ca95b08ce5d747b700a4bee7c5"
server_key_version: "encryptionkey@0"
I20260811 11:14:05.833215 16925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:05.854986 16925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:05.857523 16925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:05.857708 16925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:05.857995 16925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:05.858345 16925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:05.858403 16925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.858448 16925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:05.858474 16925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.865111 16925 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36239
I20260811 11:14:05.865195 17071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36239 every 8 connection(s)
I20260811 11:14:05.865530 16925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:05.866700 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.866253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:41609)
0811 11:14:05.866450 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.866529 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
W20260811 11:14:05.867238 17072 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:37699 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:37699: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
I20260811 11:14:05.867262 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 16925
I20260811 11:14:05.867573 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.867458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:38109)
0811 11:14:05.867514 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.867545 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:14:05.867573 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:46813
--local_ip_for_outbound_sockets=127.29.145.3
--tserver_master_addrs=127.29.145.62:37699
--webserver_port=45269
--webserver_interface=127.29.145.3
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:14:05.868007 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.867895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:50477)
0811 11:14:05.867946 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.867977 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 11:14:05.868206 17072 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:14:05.951670 17076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:05.951907 17076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:05.951946 17076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:05.953069 17076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:05.953137 17076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:05.953184 17076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:05.954686 17076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:46813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=45269
--tserver_master_addrs=127.29.145.62:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:05.954958 17076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:05.955214 17076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:05.961453 17082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.961462 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:05.961462 17083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:05.961797 17076 server_base.cc:1034] running on GCE node
I20260811 11:14:05.962108 17076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:05.962430 17076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:05.963657 17076 hybrid_clock.cc:648] HybridClock initialized: now 1786446845963619 us; error 48 us; skew 500 ppm
Aug 11 11:14:05 dist-test-slave-69rg krb5kdc[15403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:05.966308 17076 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:14:05.966709 17076 webserver.cc:460] Webserver started at http://127.29.145.3:45269/ using document root <none> and password file <none>
I20260811 11:14:05.966949 17076 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:05.967022 17076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:05.968395 17076 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:05.969018 17093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.969228 17076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:05.969344 17076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "e0a88b46d3c944f590d2804a8fbae9cc"
format_stamp: "Formatted at 2026-08-11 11:13:23 on dist-test-slave-69rg"
server_key: "1c0c9f1ebb919a0bdf0ea34ca1623f91"
server_key_iv: "197ea9ab9f1f06d89a2587fd16dd0fc2"
server_key_version: "encryptionkey@0"
I20260811 11:14:05.969702 17076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:05.981019 17076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:05.983305 17076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:05.983490 17076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:05.983717 17076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:05.984018 17076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:05.984073 17076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.984174 17076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:05.984206 17076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:05.990244 17076 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:46813
I20260811 11:14:05.990352 17225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:46813 every 8 connection(s)
I20260811 11:14:05.990679 17076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:05.991817 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.991433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:35491)
0811 11:14:05.991605 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.991680 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
W20260811 11:14:05.992298 17226 heartbeater.cc:644] Failed to heartbeat to 127.29.145.62:37699 (0 consecutive failures): Network error: Failed to ping master at 127.29.145.62:37699: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
I20260811 11:14:05.992617 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.992520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:56367)
0811 11:14:05.992569 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.992597 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 11:14:05.992893 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:05.992819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:56805)
0811 11:14:05.992858 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:05.992879 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 11:14:05.993002 17226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:14:05.993733 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17076
I20260811 11:14:06.785552 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:06.785213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:42557)
0811 11:14:06.785406 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:06.785459 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
I20260811 11:14:06.868819 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:06.868562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:42285)
0811 11:14:06.868708 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:06.868749 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260811 11:14:06.993709 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:06.993429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:45849)
0811 11:14:06.993600 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:06.993645 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260811 11:14:07.578518 17233 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:07.578784 17233 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:07.579298 17233 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
Aug 11 11:14:07 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.3@KRBTEST.COM
Aug 11 11:14:07 dist-test-slave-69rg krb5kdc[15403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446845, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.2@KRBTEST.COM
I20260811 11:14:07.599300 17240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.579829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:39937 (local address 127.29.145.3:46813)
0811 11:14:07.580004 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:07.580021 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:07.580055 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:14:07.580836 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:07.580840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:07.580917 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:07.581108 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:07.582095 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.583214 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.584414 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.584937 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.585087 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:07.594612 (+ 9525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:07.594620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:07.594621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:07.594640 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:07.597121 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:07.597720 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:07.597731 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:07.597732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:07.597927 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:07.598217 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:07.598219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:07.598219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:07.598556 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:07.598658 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:07.598985 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 11:14:07.599154 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260811 11:14:07.599283 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.579577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:46813 (local address 127.29.145.1:39937)
0811 11:14:07.579764 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:07.579779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:07.579868 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:07.581153 (+ 1285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:07.581157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:07.581611 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:07.581953 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:07.581966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.583348 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.583350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:07.584281 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:07.584284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.585051 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.585053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:07.585179 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:07.591963 (+ 6784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:07.591982 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:07.594387 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:07.597251 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:07.597253 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:07.597253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:07.597601 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:07.598004 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:07.598005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:07.598005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:07.598143 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:07.598641 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:07.598649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:07.598797 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 11:14:07.598909 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:14:07.599166 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":47,"thread_start_us":88,"threads_started":1}
I20260811 11:14:07.599418 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.579661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36239 (local address 127.29.145.1:37591)
0811 11:14:07.579813 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:07.579822 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:07.579869 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:07.581156 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:07.581158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:07.581611 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:07.581952 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:07.581963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.583354 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.583355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:07.584281 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:07.584284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.585056 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.585057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:07.585179 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:07.591962 (+ 6783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:07.591980 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:07.594534 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:07.597213 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:07.597226 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:07.597230 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:07.597618 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:07.598036 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:07.598037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:07.598038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:07.598145 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:07.598654 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:07.598655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:07.598798 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 11:14:07.598914 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:14:07.599262 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":236,"thread_start_us":71,"threads_started":1}
I20260811 11:14:07.599339 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.579792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:37591 (local address 127.29.145.2:36239)
0811 11:14:07.580021 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:07.580031 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:07.580055 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:14:07.580937 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:07.580940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:07.580994 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:07.581155 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:07.582096 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.583213 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.584413 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:07.584917 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:07.585094 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:07.594707 (+ 9613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:07.594713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:07.594713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:07.594728 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:07.597071 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:07.597753 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:07.597757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:07.597758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:07.597949 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:07.598229 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:07.598230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:07.598231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:07.598559 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:07.598662 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:07.598999 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 11:14:07.599194 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
W20260811 11:14:07.600416 16782 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
W20260811 11:14:07.600908 16781 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:14:07.600991 16781 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ad1c4f45244aabbf9f8607667afcc9; no voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc
I20260811 11:14:07.601217 17233 raft_consensus.cc:2726] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:14:07.786686 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.786327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:42549)
0811 11:14:07.786446 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:07.786508 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 11:14:07.870222 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.869642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:41273)
0811 11:14:07.870051 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:07.870130 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":133,"threads_started":1}
I20260811 11:14:07.994623 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:07.994303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:39355)
0811 11:14:07.994501 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:07.994555 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":107,"threads_started":1}
I20260811 11:14:08.787855 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:08.787460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:56111)
0811 11:14:08.787697 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:08.787758 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":110,"threads_started":1}
I20260811 11:14:08.871244 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:08.870928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:55833)
0811 11:14:08.871120 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:08.871175 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":106,"threads_started":1}
I20260811 11:14:08.995555 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:08.995232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:54811)
0811 11:14:08.995424 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:08.995473 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260811 11:14:09.270272 17247 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:09.270424 17247 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:09.270675 17247 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
W20260811 11:14:09.271308 16782 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
W20260811 11:14:09.271411 16781 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:14:09.271497 16782 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ad1c4f45244aabbf9f8607667afcc9; no voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc
I20260811 11:14:09.271608 17247 raft_consensus.cc:2726] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:14:09.789050 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:09.788699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:38075)
0811 11:14:09.788898 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:09.788956 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260811 11:14:09.872239 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:09.871914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:46143)
0811 11:14:09.872127 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:09.872173 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":144,"threads_started":1}
I20260811 11:14:09.996691 17252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:09.996313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:41369)
0811 11:14:09.996565 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:09.996626 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":145,"threads_started":1}
I20260811 11:14:10.790113 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:10.789759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:43331)
0811 11:14:10.790006 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:10.790046 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":178,"threads_started":1}
I20260811 11:14:10.873231 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:10.872883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:36729)
0811 11:14:10.873104 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:10.873161 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260811 11:14:10.924082 17256 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:10.924407 17256 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:10.924711 17256 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
W20260811 11:14:10.925272 16781 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
W20260811 11:14:10.925395 16782 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:14:10.925467 16782 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ad1c4f45244aabbf9f8607667afcc9; no voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc
I20260811 11:14:10.925547 17256 raft_consensus.cc:2726] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:14:10.997593 17257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:10.997289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:34739)
0811 11:14:10.997483 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:10.997524 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":124,"threads_started":1}
I20260811 11:14:11.791118 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:11.790814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:42437)
0811 11:14:11.790992 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:11.791048 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260811 11:14:11.874440 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:11.874127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:40145)
0811 11:14:11.874317 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:11.874371 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260811 11:14:11.998517 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:11.998215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:46447)
0811 11:14:11.998397 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:11.998444 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":169,"threads_started":1}
I20260811 11:14:12.792133 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:12.791787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:37175)
0811 11:14:12.792008 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:12.792063 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260811 11:14:12.875494 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:12.875170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:42541)
0811 11:14:12.875368 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:12.875424 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
I20260811 11:14:12.999442 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:12.999080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:33465)
0811 11:14:12.999277 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:12.999322 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":120,"threads_started":1}
I20260811 11:14:13.268966 17264 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:13.269168 17264 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:13.269590 17264 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
W20260811 11:14:13.270125 16782 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
W20260811 11:14:13.270206 16781 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:14:13.270295 16781 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ad1c4f45244aabbf9f8607667afcc9; no voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc
I20260811 11:14:13.270406 17264 raft_consensus.cc:2726] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:14:13.793022 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:13.792732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:56827)
0811 11:14:13.792912 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:13.792954 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260811 11:14:13.876446 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:13.876150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:34183)
0811 11:14:13.876319 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:13.876374 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260811 11:14:14.000299 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:13.999996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:49005)
0811 11:14:14.000187 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:14.000234 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260811 11:14:14.794075 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:14.793739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:45423)
0811 11:14:14.793955 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:14.794004 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":138,"threads_started":1}
I20260811 11:14:14.877374 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:14.877039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:42231)
0811 11:14:14.877252 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:14.877299 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
I20260811 11:14:15.001442 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:15.001108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:48321)
0811 11:14:15.001316 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:15.001372 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":97,"threads_started":1}
I20260811 11:14:15.795159 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:15.794801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:36719)
0811 11:14:15.795028 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:15.795090 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260811 11:14:15.878436 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:15.878112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:39591)
0811 11:14:15.878325 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:15.878369 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":140,"threads_started":1}
I20260811 11:14:16.002898 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:16.002543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:46995)
0811 11:14:16.002765 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:16.002825 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260811 11:14:16.796186 17276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:16.795861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:58543)
0811 11:14:16.796064 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:16.796115 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":91,"threads_started":1}
I20260811 11:14:16.879328 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:16.879039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:55753)
0811 11:14:16.879214 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:16.879259 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260811 11:14:17.003909 17278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:17.003597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:59627)
0811 11:14:17.003774 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:17.003830 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260811 11:14:17.470788 17279 raft_consensus.cc:482] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:17.470932 17279 raft_consensus.cc:504] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d94e62a094409e8e18f0548489e3a8" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36239 } } peers { permanent_uuid: "e0a88b46d3c944f590d2804a8fbae9cc" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 46813 } } peers { permanent_uuid: "f7ad1c4f45244aabbf9f8607667afcc9" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 44569 } }
I20260811 11:14:17.471253 17279 leader_election.cc:288] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239), e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813)
W20260811 11:14:17.471765 16781 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 31d94e62a094409e8e18f0548489e3a8 (127.29.145.2:36239): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
W20260811 11:14:17.471881 16782 leader_election.cc:341] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e0a88b46d3c944f590d2804a8fbae9cc (127.29.145.3:46813): Not found: Tablet not found: 727839690df6403c9d0e1bdb5cd4b60f
I20260811 11:14:17.471938 16782 leader_election.cc:302] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7ad1c4f45244aabbf9f8607667afcc9; no voters: 31d94e62a094409e8e18f0548489e3a8, e0a88b46d3c944f590d2804a8fbae9cc
I20260811 11:14:17.472020 17279 raft_consensus.cc:2726] T 727839690df6403c9d0e1bdb5cd4b60f P f7ad1c4f45244aabbf9f8607667afcc9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:14:17.797128 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:17.796748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:56529)
0811 11:14:17.797003 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:17.797051 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":125,"threads_started":1}
I20260811 11:14:17.880123 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:17.879831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:42469)
0811 11:14:17.880011 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:17.880055 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260811 11:14:18.005223 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:18.004852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:44953)
0811 11:14:18.005090 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:18.005153 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":106,"threads_started":1}
I20260811 11:14:18.798110 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:18.797732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:42947)
0811 11:14:18.797980 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:18.798030 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":164,"threads_started":1}
I20260811 11:14:18.881004 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:18.880712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:53713)
0811 11:14:18.880898 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:18.880935 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":124,"threads_started":1}
I20260811 11:14:19.006636 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:19.006296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:40405)
0811 11:14:19.006500 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:19.006567 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260811 11:14:19.799027 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:19.798743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:37539)
0811 11:14:19.798911 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:19.798956 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260811 11:14:19.881901 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:19.881527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:45123)
0811 11:14:19.881734 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:19.881781 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260811 11:14:20.007920 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:20.007519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:57247)
0811 11:14:20.007773 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:20.007848 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":116,"threads_started":1}
I20260811 11:14:20.800134 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:20.799804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.1:53649)
0811 11:14:20.800011 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:20.800063 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260811 11:14:20.883026 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:20.882736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.2:57279)
0811 11:14:20.882926 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:20.882964 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
I20260811 11:14:21.002323 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16716
I20260811 11:14:21.009168 17293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:21.008843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:37699 (local address 127.29.145.3:59129)
0811 11:14:21.009041 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:21.009093 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.62:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
I20260811 11:14:21.015544 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 16925
I20260811 11:14:21.027096 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17076
2026-08-11T11:14:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59046 ms)
[----------] 30 tests from SecurityITest (249805 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:21 dist-test-slave-69rg krb5kdc[17296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:21 dist-test-slave-69rg krb5kdc[17296](info): set up 2 sockets
Aug 11 11:14:21 dist-test-slave-69rg krb5kdc[17296](info): commencing operation
krb5kdc: starting...
W20260811 11:14:22.196038 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:22.334589 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43303
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:43303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:22.422104 17312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:22.422338 17312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:22.422361 17312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:22.423481 17312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:22.423520 17312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:22.423534 17312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:22.423547 17312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:22.423559 17312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:22.425036 17312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:43303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:22.425272 17312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:22.425451 17312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:14:22.431490 17312 server_base.cc:1034] running on GCE node
W20260811 11:14:22.431452 17322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.431460 17320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.431468 17319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:22.431998 17312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:22.432322 17312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:22.433482 17312 hybrid_clock.cc:648] HybridClock initialized: now 1786446862433426 us; error 74 us; skew 500 ppm
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:22.436394 17312 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:22.436874 17312 webserver.cc:460] Webserver started at http://127.29.145.62:33331/ using document root <none> and password file <none>
I20260811 11:14:22.437122 17312 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:22.437180 17312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:22.437319 17312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:22.438515 17312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "af01bc7e69e84515a035f45d3f84276a"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "d51356c8855909729b2beeb1414c6f53"
server_key_iv: "8c679bbe442c6870c045bfa24f22ce3f"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.438906 17312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "af01bc7e69e84515a035f45d3f84276a"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "d51356c8855909729b2beeb1414c6f53"
server_key_iv: "8c679bbe442c6870c045bfa24f22ce3f"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.440351 17312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:14:22.441146 17329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.441332 17312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:22.441431 17312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "af01bc7e69e84515a035f45d3f84276a"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "d51356c8855909729b2beeb1414c6f53"
server_key_iv: "8c679bbe442c6870c045bfa24f22ce3f"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.441506 17312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:22.460929 17312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:22.463215 17312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:22.463407 17312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:22.467975 17312 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:43303
I20260811 11:14:22.468010 17386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:43303 every 8 connection(s)
I20260811 11:14:22.468430 17312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:22.469337 17389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:22.471139 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17312
I20260811 11:14:22.471235 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:22.471458 30276 external_mini_cluster.cc:1428] Setting key ff397ce2af732358b101c49b6b664579
I20260811 11:14:22.472034 17389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: Bootstrap starting.
I20260811 11:14:22.472785 17389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:22.473116 17389 log.cc:824] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:22.473891 17389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: No bootstrap required, opened a new log
I20260811 11:14:22.476567 17389 raft_consensus.cc:348] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } }
I20260811 11:14:22.476749 17389 raft_consensus.cc:374] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:22.476785 17389 raft_consensus.cc:729] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af01bc7e69e84515a035f45d3f84276a, State: Initialized, Role: FOLLOWER
I20260811 11:14:22.476910 17389 consensus_queue.cc:260] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } }
I20260811 11:14:22.476987 17389 raft_consensus.cc:388] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:22.477018 17389 raft_consensus.cc:482] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:22.477064 17389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:22.477804 17389 raft_consensus.cc:504] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } }
I20260811 11:14:22.477945 17389 leader_election.cc:302] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af01bc7e69e84515a035f45d3f84276a; no voters:
I20260811 11:14:22.478186 17389 leader_election.cc:288] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:22.478209 17394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:22.478339 17394 raft_consensus.cc:686] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [term 1 LEADER]: Becoming Leader. State: Replica: af01bc7e69e84515a035f45d3f84276a, State: Running, Role: LEADER
I20260811 11:14:22.478487 17394 consensus_queue.cc:237] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } }
I20260811 11:14:22.478498 17389 sys_catalog.cc:564] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:22.478971 17395 sys_catalog.cc:455] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af01bc7e69e84515a035f45d3f84276a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } } }
I20260811 11:14:22.478981 17396 sys_catalog.cc:455] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [sys.catalog]: SysCatalogTable state changed. Reason: New leader af01bc7e69e84515a035f45d3f84276a. Latest consensus state: current_term: 1 leader_uuid: "af01bc7e69e84515a035f45d3f84276a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af01bc7e69e84515a035f45d3f84276a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43303 } } }
I20260811 11:14:22.479070 17395 sys_catalog.cc:458] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:22.479094 17396 sys_catalog.cc:458] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:22.479313 17405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:22.479846 17405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:22.481974 17405 catalog_manager.cc:1347] Generated new cluster ID: 582fc0a8e0264e0eb5c301ac1833af85
I20260811 11:14:22.482033 17405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:22.484261 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:22.472048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38366 (local address 127.29.145.62:43303)
0811 11:14:22.472255 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:22.472264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:22.472294 (+ 30us) server_negotiation.cc:407] Connection header received
0811 11:14:22.472936 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:22.472948 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:22.473029 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:22.473201 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:22.473924 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.475363 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.476538 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.477175 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.477295 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:22.480634 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:22.480638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:22.480638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:22.480651 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:22.482632 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.483021 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.483024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.483025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.483125 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.483354 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.483355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.483355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.483597 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:22.483788 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:22.483889 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 11:14:22.484033 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260811 11:14:22.496995 17405 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:22.497620 17405 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:22.501515 17405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: Generated new TSK 0
I20260811 11:14:22.501741 17405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:14:22.536532 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:22.638145 17422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:22.638358 17422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:22.638383 17422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:22.639523 17422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:22.639567 17422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:22.639595 17422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:22.641073 17422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:22.641340 17422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:22.641588 17422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:22.647089 17428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.647078 17429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.647089 17431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:22.647352 17422 server_base.cc:1034] running on GCE node
I20260811 11:14:22.647598 17422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:22.647809 17422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:22.648958 17422 hybrid_clock.cc:648] HybridClock initialized: now 1786446862648930 us; error 46 us; skew 500 ppm
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:22.651688 17422 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:14:22.652122 17422 webserver.cc:460] Webserver started at http://127.29.145.1:35037/ using document root <none> and password file <none>
I20260811 11:14:22.652371 17422 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:22.652420 17422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:22.652539 17422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:22.653612 17422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "cd9b5250400b4e7c935e8ac60b507395"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "78b1eb136507e8a36b45d59c7f151995"
server_key_iv: "7b1bab499509b69a96e0c62be816369e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.654057 17422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "cd9b5250400b4e7c935e8ac60b507395"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "78b1eb136507e8a36b45d59c7f151995"
server_key_iv: "7b1bab499509b69a96e0c62be816369e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.655485 17422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:22.656358 17438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.656594 17422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:22.656683 17422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "cd9b5250400b4e7c935e8ac60b507395"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "78b1eb136507e8a36b45d59c7f151995"
server_key_iv: "7b1bab499509b69a96e0c62be816369e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.656778 17422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:22.668907 17422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:22.671216 17422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:22.671397 17422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:22.671653 17422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:22.672008 17422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:22.672070 17422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.672107 17422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:22.672138 17422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.678413 17422 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:39755
I20260811 11:14:22.678488 17570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:39755 every 8 connection(s)
I20260811 11:14:22.678871 17422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:14:22.683462 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17422
I20260811 11:14:22.683578 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:22.683802 30276 external_mini_cluster.cc:1428] Setting key 529bc1394f2dc289416fffb6553f33bf
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:22.689244 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:22.679675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:57563 (local address 127.29.145.62:43303)
0811 11:14:22.679738 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:22.679740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:22.679999 (+ 259us) server_negotiation.cc:407] Connection header received
0811 11:14:22.680543 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:22.680544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:22.680558 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:22.680615 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:22.681587 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.682202 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.683217 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.683542 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.683629 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:22.686239 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:22.686240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:22.686240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:22.686249 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:22.687739 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.688267 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.688268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.688268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.688319 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.688606 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.688607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.688607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.688723 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:22.688776 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:22.689103 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 11:14:22.689176 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:14:22.689602 17573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:22.679631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43303 (local address 127.29.145.1:57563)
0811 11:14:22.679858 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:22.679904 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:22.680433 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:22.680694 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:22.680698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:22.681287 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:22.681480 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:22.681485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.682306 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.682307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:22.683116 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:22.683118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.683715 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.683716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:22.683860 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:22.684466 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:22.684475 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:22.686135 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:22.687828 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:22.687836 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:22.687838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:22.688190 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:22.688387 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:22.688387 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:22.688388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:22.688543 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:22.688792 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:22.688799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:22.688969 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 11:14:22.689083 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 11:14:22.689303 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
I20260811 11:14:22.690255 17571 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43303
I20260811 11:14:22.690387 17571 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:22.690577 17571 heartbeater.cc:511] Master 127.29.145.62:43303 requested a full tablet report, sending...
I20260811 11:14:22.691157 17348 ts_manager.cc:194] Registered new tserver with Master: cd9b5250400b4e7c935e8ac60b507395 (127.29.145.1:39755)
I20260811 11:14:22.691800 17348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:57563
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:14:22.713250 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:22.801268 17577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:22.801463 17577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:22.801486 17577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:22.802630 17577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:22.802672 17577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:22.802698 17577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:22.804167 17577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:22.804422 17577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:22.804665 17577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:22.809881 17583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:22.810007 17577 server_base.cc:1034] running on GCE node
W20260811 11:14:22.809897 17584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.809911 17586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:22.810376 17577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:22.810622 17577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:22.811798 17577 hybrid_clock.cc:648] HybridClock initialized: now 1786446862811794 us; error 42 us; skew 500 ppm
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:22.814484 17577 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:14:22.814903 17577 webserver.cc:460] Webserver started at http://127.29.145.2:44125/ using document root <none> and password file <none>
I20260811 11:14:22.815110 17577 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:22.815160 17577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:22.815250 17577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:22.816246 17577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "04c8b09baeb843289a4327c596f6a52d"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "2b642f65a09b7921e7379b74f4efb723"
server_key_iv: "3c06fa3871396e7b6abf14e3fb6e5acf"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.816558 17577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "04c8b09baeb843289a4327c596f6a52d"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "2b642f65a09b7921e7379b74f4efb723"
server_key_iv: "3c06fa3871396e7b6abf14e3fb6e5acf"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.817986 17577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:22.818817 17593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.819131 17577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:22.819247 17577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "04c8b09baeb843289a4327c596f6a52d"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "2b642f65a09b7921e7379b74f4efb723"
server_key_iv: "3c06fa3871396e7b6abf14e3fb6e5acf"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.819305 17577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:22.839176 17577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:22.841693 17577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:22.841868 17577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:22.842095 17577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:22.842363 17577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:22.842396 17577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.842425 17577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:22.842445 17577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.848898 17577 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:35757
I20260811 11:14:22.848958 17728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:35757 every 8 connection(s)
I20260811 11:14:22.849282 17577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:22.849510 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17577
I20260811 11:14:22.849637 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:22.849886 30276 external_mini_cluster.cc:1428] Setting key 014e054f8ab1530bcd1db15edec59d09
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:22.860711 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:22.850432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:38437 (local address 127.29.145.62:43303)
0811 11:14:22.850498 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:22.850500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:22.850514 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:14:22.851277 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:22.851278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:22.851289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:22.851349 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:22.852708 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.853214 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.854667 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.854875 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.854979 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:22.857666 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:22.857667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:22.857668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:22.857675 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:22.859164 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.859652 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.859653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.859654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.859706 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:22.859991 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:22.859991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:22.859992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:22.860099 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:22.860149 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:22.860497 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 11:14:22.860565 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:14:22.860929 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:22.850185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43303 (local address 127.29.145.2:38437)
0811 11:14:22.850374 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:22.850427 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:22.851151 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:22.851419 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:22.851424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:22.852262 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:22.852542 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:22.852549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.853343 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.853345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:22.854563 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:22.854566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:22.855001 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:22.855003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:22.855140 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:22.855736 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:22.855746 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:22.857544 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:22.859255 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:22.859263 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:22.859265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:22.859579 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:22.859779 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:22.859780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:22.859781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:22.859913 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:22.860167 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:22.860174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:22.860347 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 11:14:22.860477 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 11:14:22.860678 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
I20260811 11:14:22.861553 17729 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43303
I20260811 11:14:22.861641 17729 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:22.861795 17729 heartbeater.cc:511] Master 127.29.145.62:43303 requested a full tablet report, sending...
I20260811 11:14:22.862241 17348 ts_manager.cc:194] Registered new tserver with Master: 04c8b09baeb843289a4327c596f6a52d (127.29.145.2:35757)
I20260811 11:14:22.862591 17348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:38437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:14:22.882027 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:22.971962 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:22.972239 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:22.972281 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:22.973453 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:22.973529 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:22.973575 17736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:22.975173 17736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:22.975481 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:22.975752 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:22.981391 17746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.981447 17744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:22.981380 17743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:22.982095 17736 server_base.cc:1034] running on GCE node
I20260811 11:14:22.982293 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:22.982498 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:22.983655 17736 hybrid_clock.cc:648] HybridClock initialized: now 1786446862983642 us; error 30 us; skew 500 ppm
Aug 11 11:14:22 dist-test-slave-69rg krb5kdc[17296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:22.986428 17736 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:14:22.986879 17736 webserver.cc:460] Webserver started at http://127.29.145.3:44697/ using document root <none> and password file <none>
I20260811 11:14:22.987140 17736 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:22.987213 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:22.987365 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:22.988445 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "7db921627a304c69bdc0768130749016"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "15c2d7727deffeebf70e9e0c28ead886"
server_key_iv: "de8df90d685c61b106333802e3cebc0e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.988838 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "7db921627a304c69bdc0768130749016"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "15c2d7727deffeebf70e9e0c28ead886"
server_key_iv: "de8df90d685c61b106333802e3cebc0e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.990293 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:22.991107 17753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:22.991317 17736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:22.991430 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "7db921627a304c69bdc0768130749016"
format_stamp: "Formatted at 2026-08-11 11:14:22 on dist-test-slave-69rg"
server_key: "15c2d7727deffeebf70e9e0c28ead886"
server_key_iv: "de8df90d685c61b106333802e3cebc0e"
server_key_version: "encryptionkey@0"
I20260811 11:14:22.991506 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:23.004230 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:23.006686 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:23.006850 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:23.007084 17736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:23.007404 17736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:23.007460 17736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.007503 17736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:23.007532 17736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.013712 17736 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:36761
I20260811 11:14:23.013777 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:36761 every 8 connection(s)
I20260811 11:14:23.014245 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:23.018034 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17736
I20260811 11:14:23.018136 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:23.018323 30276 external_mini_cluster.cc:1428] Setting key 3fe8fd5857c5d4c1dd24b42602c0f2ac
Aug 11 11:14:23 dist-test-slave-69rg krb5kdc[17296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446862, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:23.024521 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.015384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:52375 (local address 127.29.145.62:43303)
0811 11:14:23.015447 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.015449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.015656 (+ 207us) server_negotiation.cc:407] Connection header received
0811 11:14:23.016130 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.016131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.016141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.016206 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:23.017117 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.017614 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.018938 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.019160 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.019239 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.021525 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:23.021527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:23.021528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:23.021535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:23.023067 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:23.023539 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:23.023541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:23.023541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:23.023605 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:23.023872 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:23.023873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:23.023873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:23.023993 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:23.024070 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.024344 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.024428 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:14:23.024729 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.015354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43303 (local address 127.29.145.3:52375)
0811 11:14:23.015516 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.015557 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.016045 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.016261 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.016264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.016845 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:23.017039 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.017043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.017708 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.017709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.018842 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.018846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.019247 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.019248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.019388 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.019950 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:23.019959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:23.021396 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:23.023204 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:23.023211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:23.023213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:23.023455 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:23.023677 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:23.023678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:23.023678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:23.023802 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:23.024071 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:23.024078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:23.024215 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.024322 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.024478 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260811 11:14:23.025278 17883 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43303
I20260811 11:14:23.025389 17883 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:23.025583 17883 heartbeater.cc:511] Master 127.29.145.62:43303 requested a full tablet report, sending...
I20260811 11:14:23.026041 17348 ts_manager.cc:194] Registered new tserver with Master: 7db921627a304c69bdc0768130749016 (127.29.145.3:36761)
I20260811 11:14:23.026451 17348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:52375
I20260811 11:14:23.030021 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:23.037869 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.031546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.29.145.62:43303)
0811 11:14:23.031626 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.031629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.031732 (+ 103us) server_negotiation.cc:407] Connection header received
0811 11:14:23.031811 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.031812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.031822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.031878 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:23.032465 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.032942 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.033683 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.034007 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.034097 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.035288 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:23.035289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:23.035290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:23.035297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:23.036637 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:23.036991 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:23.036992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:23.036993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:23.037050 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:23.037278 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:23.037279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:23.037279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:23.037435 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:23.037547 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.037658 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.037741 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:14:23.039498 17348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:23.041083 17348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:23.047674 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.29.145.2:35757)
0811 11:14:23.044033 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.044037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.044273 (+ 236us) server_negotiation.cc:407] Connection header received
0811 11:14:23.044353 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.044356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.044373 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.044454 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:23.044575 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045716 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.046709 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.047228 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.047318 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.047382 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.047547 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.047593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
I20260811 11:14:23.047672 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35757 (local address 127.0.0.1:57012)
0811 11:14:23.044128 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.044138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.044215 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.044461 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.044462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.044464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:23.044514 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.044515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045850 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.045851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.046606 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.046615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.047319 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.047320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.047409 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.047424 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.047528 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.047587 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":64,"threads_started":1}
I20260811 11:14:23.048067 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39755 (local address 127.0.0.1:47944)
0811 11:14:23.043887 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.043930 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.044019 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.044395 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.044396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.044399 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:23.044498 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.044500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045577 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.045577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.046836 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.046840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.047823 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.047824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.047884 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.047887 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.047943 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.047985 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":46,"threads_started":1}
I20260811 11:14:23.048166 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.29.145.1:39755)
0811 11:14:23.043820 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.043824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.044027 (+ 203us) server_negotiation.cc:407] Connection header received
0811 11:14:23.044186 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.044191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.044212 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.044296 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:23.044599 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045479 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.046938 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.047667 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.047714 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.047760 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.047977 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.048021 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":101,"threads_started":1}
I20260811 11:14:23.048475 17901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36761 (local address 127.0.0.1:34666)
0811 11:14:23.044006 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.044015 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.044067 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.044381 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.044384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.044386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:23.044457 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.044459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045790 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.045791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.047251 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.047254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.048238 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.048239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.048323 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.048327 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.048371 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.048413 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":320,"thread_start_us":89,"threads_started":1}
I20260811 11:14:23.048527 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.043667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.29.145.3:36761)
0811 11:14:23.043880 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.043883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.044140 (+ 257us) server_negotiation.cc:407] Connection header received
0811 11:14:23.044189 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.044192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.044210 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.044282 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:23.044538 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.045679 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.047349 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.048074 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.048141 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.048201 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.048402 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.048445 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260811 11:14:23.049158 17652 tablet_service.cc:1467] Processing CreateTablet for tablet a438fceb7e57461c809322cd215a4a72 (DEFAULT_TABLE table=test-table [id=21d2a1bf126c485aa912d5b1c7f5f3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:23.049154 17488 tablet_service.cc:1467] Processing CreateTablet for tablet a438fceb7e57461c809322cd215a4a72 (DEFAULT_TABLE table=test-table [id=21d2a1bf126c485aa912d5b1c7f5f3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:23.049556 17488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a438fceb7e57461c809322cd215a4a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:23.049546 17652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a438fceb7e57461c809322cd215a4a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:23.053239 17904 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Bootstrap starting.
I20260811 11:14:23.053239 17903 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Bootstrap starting.
I20260811 11:14:23.054025 17904 tablet_bootstrap.cc:654] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:23.054025 17903 tablet_bootstrap.cc:654] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:23.054169 17807 tablet_service.cc:1467] Processing CreateTablet for tablet a438fceb7e57461c809322cd215a4a72 (DEFAULT_TABLE table=test-table [id=21d2a1bf126c485aa912d5b1c7f5f3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:23.054527 17807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a438fceb7e57461c809322cd215a4a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:23.054723 17903 log.cc:824] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:23.054723 17904 log.cc:824] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:23.055538 17903 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: No bootstrap required, opened a new log
I20260811 11:14:23.055588 17903 ts_tablet_manager.cc:1397] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:23.055646 17904 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: No bootstrap required, opened a new log
I20260811 11:14:23.055711 17904 ts_tablet_manager.cc:1397] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:14:23.057024 17903 raft_consensus.cc:348] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.057155 17903 raft_consensus.cc:374] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:23.057209 17903 raft_consensus.cc:729] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd9b5250400b4e7c935e8ac60b507395, State: Initialized, Role: FOLLOWER
I20260811 11:14:23.057356 17903 consensus_queue.cc:260] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.057670 17571 heartbeater.cc:503] Master 127.29.145.62:43303 was elected leader, sending a full tablet report...
I20260811 11:14:23.057667 17903 ts_tablet_manager.cc:1428] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:23.058136 17908 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Bootstrap starting.
I20260811 11:14:23.058068 17904 raft_consensus.cc:348] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.058279 17904 raft_consensus.cc:374] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:23.058336 17904 raft_consensus.cc:729] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c8b09baeb843289a4327c596f6a52d, State: Initialized, Role: FOLLOWER
I20260811 11:14:23.058451 17904 consensus_queue.cc:260] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.058724 17904 ts_tablet_manager.cc:1428] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:14:23.058849 17729 heartbeater.cc:503] Master 127.29.145.62:43303 was elected leader, sending a full tablet report...
I20260811 11:14:23.059078 17908 tablet_bootstrap.cc:654] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:23.059979 17908 log.cc:824] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:23.060981 17908 tablet_bootstrap.cc:492] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: No bootstrap required, opened a new log
I20260811 11:14:23.061168 17908 ts_tablet_manager.cc:1397] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:14:23.062729 17908 raft_consensus.cc:348] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.062942 17908 raft_consensus.cc:374] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:23.062994 17908 raft_consensus.cc:729] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7db921627a304c69bdc0768130749016, State: Initialized, Role: FOLLOWER
I20260811 11:14:23.063128 17908 consensus_queue.cc:260] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.063431 17908 ts_tablet_manager.cc:1428] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 11:14:23.063485 17883 heartbeater.cc:503] Master 127.29.145.62:43303 was elected leader, sending a full tablet report...
W20260811 11:14:23.100060 17731 tablet.cc:2369] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:14:23.179733 17572 tablet.cc:2369] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:14:23.265213 17884 tablet.cc:2369] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:23.335813 17909 raft_consensus.cc:482] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:23.336153 17909 raft_consensus.cc:504] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.336696 17909 leader_election.cc:288] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7db921627a304c69bdc0768130749016 (127.29.145.3:36761), cd9b5250400b4e7c935e8ac60b507395 (127.29.145.1:39755)
I20260811 11:14:23.341086 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.337032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:59937 (local address 127.29.145.1:39755)
0811 11:14:23.337086 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.337089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.337252 (+ 163us) server_negotiation.cc:407] Connection header received
0811 11:14:23.337311 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.337312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.337323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.337378 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:23.337545 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.338392 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340071 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.340655 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340746 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.340758 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.340953 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.341002 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:14:23.341086 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.337066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:48771 (local address 127.29.145.3:36761)
0811 11:14:23.337139 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.337142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.337151 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:23.337193 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.337194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.337203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.337287 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:23.337541 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.338392 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340066 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.340673 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340746 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.340758 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.340953 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.341009 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:14:23.341099 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.336943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36761 (local address 127.29.145.2:48771)
0811 11:14:23.337015 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.337029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.337081 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.337312 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.337313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.337316 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:23.337427 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.337429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.338499 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.338501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.339950 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.339953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.340759 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.340872 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.340875 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.340927 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.340994 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":166}
I20260811 11:14:23.341107 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.336959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39755 (local address 127.29.145.2:59937)
0811 11:14:23.337160 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.337170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.337237 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.337400 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.337401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.337402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:23.337468 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.337469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.338499 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.338501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.339950 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.339953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.340759 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.340761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.340860 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.340865 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.340927 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.340994 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":51,"thread_start_us":89,"threads_started":1}
I20260811 11:14:23.341609 17830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a438fceb7e57461c809322cd215a4a72" candidate_uuid: "04c8b09baeb843289a4327c596f6a52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db921627a304c69bdc0768130749016" is_pre_election: true
I20260811 11:14:23.341856 17830 raft_consensus.cc:2454] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04c8b09baeb843289a4327c596f6a52d in term 0.
I20260811 11:14:23.341850 17514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a438fceb7e57461c809322cd215a4a72" candidate_uuid: "04c8b09baeb843289a4327c596f6a52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd9b5250400b4e7c935e8ac60b507395" is_pre_election: true
I20260811 11:14:23.342061 17514 raft_consensus.cc:2454] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04c8b09baeb843289a4327c596f6a52d in term 0.
I20260811 11:14:23.342177 17594 leader_election.cc:302] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c8b09baeb843289a4327c596f6a52d, 7db921627a304c69bdc0768130749016; no voters:
I20260811 11:14:23.342401 17909 raft_consensus.cc:2781] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:23.342448 17909 raft_consensus.cc:482] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:23.342465 17909 raft_consensus.cc:3037] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:23.343355 17909 raft_consensus.cc:504] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.343536 17909 leader_election.cc:288] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [CANDIDATE]: Term 1 election: Requested vote from peers 7db921627a304c69bdc0768130749016 (127.29.145.3:36761), cd9b5250400b4e7c935e8ac60b507395 (127.29.145.1:39755)
I20260811 11:14:23.343712 17830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a438fceb7e57461c809322cd215a4a72" candidate_uuid: "04c8b09baeb843289a4327c596f6a52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7db921627a304c69bdc0768130749016"
I20260811 11:14:23.343763 17514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a438fceb7e57461c809322cd215a4a72" candidate_uuid: "04c8b09baeb843289a4327c596f6a52d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd9b5250400b4e7c935e8ac60b507395"
I20260811 11:14:23.343843 17514 raft_consensus.cc:3037] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:23.343842 17830 raft_consensus.cc:3037] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:23.344870 17514 raft_consensus.cc:2454] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04c8b09baeb843289a4327c596f6a52d in term 1.
I20260811 11:14:23.344866 17830 raft_consensus.cc:2454] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04c8b09baeb843289a4327c596f6a52d in term 1.
I20260811 11:14:23.345049 17594 leader_election.cc:302] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04c8b09baeb843289a4327c596f6a52d, 7db921627a304c69bdc0768130749016; no voters:
I20260811 11:14:23.345144 17909 raft_consensus.cc:2781] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:23.345361 17909 raft_consensus.cc:686] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 1 LEADER]: Becoming Leader. State: Replica: 04c8b09baeb843289a4327c596f6a52d, State: Running, Role: LEADER
I20260811 11:14:23.345486 17909 consensus_queue.cc:237] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } }
I20260811 11:14:23.346565 17348 catalog_manager.cc:5495] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d reported cstate change: term changed from 0 to 1, leader changed from <none> to 04c8b09baeb843289a4327c596f6a52d (127.29.145.2). New cstate: current_term: 1 leader_uuid: "04c8b09baeb843289a4327c596f6a52d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04c8b09baeb843289a4327c596f6a52d" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 35757 } health_report { overall_health: HEALTHY } } }
I20260811 11:14:23.402546 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.399782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.29.145.2:35757)
0811 11:14:23.399869 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.399872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.399881 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:23.399937 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.399938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.399961 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.400048 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:23.400222 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.400788 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.401614 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.401840 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.401923 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.402117 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:23.402251 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:23.402354 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.402381 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.402407 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:14:23.404973 17830 raft_consensus.cc:1261] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 1 FOLLOWER]: Refusing update from remote peer 04c8b09baeb843289a4327c596f6a52d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:23.405390 17514 raft_consensus.cc:1261] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 1 FOLLOWER]: Refusing update from remote peer 04c8b09baeb843289a4327c596f6a52d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:23.405442 17913 consensus_queue.cc:1035] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7db921627a304c69bdc0768130749016" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:23.405797 17913 consensus_queue.cc:1035] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd9b5250400b4e7c935e8ac60b507395" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:23.408619 17916 mvcc.cc:205] Tried to move back new op lower bound from 7317286352503291904 to 7317286352263901184. Current Snapshot: MvccSnapshot[applied={T|T < 7317286352503291904}]
I20260811 11:14:23.409618 17917 mvcc.cc:205] Tried to move back new op lower bound from 7317286352503291904 to 7317286352263901184. Current Snapshot: MvccSnapshot[applied={T|T < 7317286352503291904 or (T in {7317286352503291904})}]
I20260811 11:14:23.412355 17346 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:23.412483 17346 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:23.413456 17346 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af01bc7e69e84515a035f45d3f84276a: Sending DeleteTablet for 3 replicas of tablet a438fceb7e57461c809322cd215a4a72
I20260811 11:14:23.413880 17807 tablet_service.cc:1514] Processing DeleteTablet for tablet a438fceb7e57461c809322cd215a4a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:23 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 11:14:23.413942 17488 tablet_service.cc:1514] Processing DeleteTablet for tablet a438fceb7e57461c809322cd215a4a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:23 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:47944
I20260811 11:14:23.414172 17928 tablet_replica.cc:330] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395: stopping tablet replica
I20260811 11:14:23.414150 17652 tablet_service.cc:1514] Processing DeleteTablet for tablet a438fceb7e57461c809322cd215a4a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:23 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:57012
I20260811 11:14:23.414295 17928 raft_consensus.cc:2229] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:23.414381 17929 tablet_replica.cc:330] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: stopping tablet replica
I20260811 11:14:23.414525 17929 raft_consensus.cc:2229] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:14:23.414525 17928 raft_consensus.cc:2258] T a438fceb7e57461c809322cd215a4a72 P cd9b5250400b4e7c935e8ac60b507395 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:23.414675 17927 tablet_replica.cc:330] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: stopping tablet replica
I20260811 11:14:23.414691 17929 raft_consensus.cc:2258] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:23.414804 17927 raft_consensus.cc:2229] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:23.414806 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17422
I20260811 11:14:23.414934 17927 raft_consensus.cc:2258] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:23.424641 17927 ts_tablet_manager.cc:1905] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:23.426883 17927 ts_tablet_manager.cc:1918] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:23.427063 17927 log.cc:1196] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/a438fceb7e57461c809322cd215a4a72
I20260811 11:14:23.427484 17927 ts_tablet_manager.cc:1939] T a438fceb7e57461c809322cd215a4a72 P 7db921627a304c69bdc0768130749016: Deleting consensus metadata
I20260811 11:14:23.428162 17330 catalog_manager.cc:4841] TS 7db921627a304c69bdc0768130749016 (127.29.145.3:36761): tablet a438fceb7e57461c809322cd215a4a72 (table test-table [id=21d2a1bf126c485aa912d5b1c7f5f3b1]) successfully deleted
W20260811 11:14:23.429096 17332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:39755 (error 108)
I20260811 11:14:23.429356 17929 ts_tablet_manager.cc:1905] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:14:23.430034 17897 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:23.429623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:39755 (local address 127.0.0.1:47956)
0811 11:14:23.429683 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.429699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.429755 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.429973 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:39755: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 11:14:23.430187 17332 catalog_manager.cc:4584] TS cd9b5250400b4e7c935e8ac60b507395 (127.29.145.1:39755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a438fceb7e57461c809322cd215a4a72: Network error: Client connection negotiation failed: client connection to 127.29.145.1:39755: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:14:23.431409 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17577
I20260811 11:14:23.431416 17929 ts_tablet_manager.cc:1918] T a438fceb7e57461c809322cd215a4a72 P 04c8b09baeb843289a4327c596f6a52d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:23.450275 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17736
I20260811 11:14:23.450356 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.450151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:35757 (local address 127.0.0.1:57038)
0811 11:14:23.450226 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.450304 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.2:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:14:23.459118 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17312
2026-08-11T11:14:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2427 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T11:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:23Z Disabled control of system clock
I20260811 11:14:23.494210 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:32837
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:32837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:23.590590 17936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:23.590873 17936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:23.590896 17936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:23.592151 17936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:23.592187 17936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:23.592199 17936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:23.592212 17936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:23.592224 17936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:23.593572 17936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:32837
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:32837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:23.593868 17936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:23.594205 17936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:23.599824 17943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.599826 17942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.599820 17945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:23.599860 17936 server_base.cc:1034] running on GCE node
I20260811 11:14:23.600335 17936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:23.600713 17936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:23.601912 17936 hybrid_clock.cc:648] HybridClock initialized: now 1786446863601897 us; error 54 us; skew 500 ppm
I20260811 11:14:23.608733 17936 webserver.cc:460] Webserver started at http://127.29.145.62:39915/ using document root <none> and password file <none>
I20260811 11:14:23.609076 17936 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:23.609156 17936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:23.609333 17936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:23.610671 17936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "b33456f1ea2148e8b21aa3669300148a"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "a1d54a556657397e910702c85bf6a3e9"
server_key_iv: "78fde82fc4eb77a161136e7db2b61da3"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.611093 17936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "b33456f1ea2148e8b21aa3669300148a"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "a1d54a556657397e910702c85bf6a3e9"
server_key_iv: "78fde82fc4eb77a161136e7db2b61da3"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.612615 17936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:14:23.613544 17953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.613739 17936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:23.613873 17936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "b33456f1ea2148e8b21aa3669300148a"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "a1d54a556657397e910702c85bf6a3e9"
server_key_iv: "78fde82fc4eb77a161136e7db2b61da3"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.613960 17936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:23.624696 17936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:23.625098 17936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:23.625283 17936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:23.630159 17936 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:32837
I20260811 11:14:23.630201 18016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:32837 every 8 connection(s)
I20260811 11:14:23.630609 17936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:23.631708 18018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:23.634527 18018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: Bootstrap starting.
I20260811 11:14:23.635215 18018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:23.635579 18018 log.cc:824] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:23.636442 18018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: No bootstrap required, opened a new log
I20260811 11:14:23.638396 18018 raft_consensus.cc:348] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } }
I20260811 11:14:23.638581 18018 raft_consensus.cc:374] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:23.638659 18018 raft_consensus.cc:729] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b33456f1ea2148e8b21aa3669300148a, State: Initialized, Role: FOLLOWER
I20260811 11:14:23.638873 18018 consensus_queue.cc:260] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } }
I20260811 11:14:23.638989 18018 raft_consensus.cc:388] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:23.639038 18018 raft_consensus.cc:482] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:23.639099 18018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:23.639708 18018 raft_consensus.cc:504] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } }
I20260811 11:14:23.639839 18018 leader_election.cc:302] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b33456f1ea2148e8b21aa3669300148a; no voters:
I20260811 11:14:23.640095 18018 leader_election.cc:288] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:23.640178 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 17936
I20260811 11:14:23.640197 18022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:23.640301 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:23.640432 18022 raft_consensus.cc:686] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [term 1 LEADER]: Becoming Leader. State: Replica: b33456f1ea2148e8b21aa3669300148a, State: Running, Role: LEADER
I20260811 11:14:23.640441 18018 sys_catalog.cc:564] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:23.640533 30276 external_mini_cluster.cc:1428] Setting key 8bff607f4c7d1354bb2d28e271dc89c3
I20260811 11:14:23.640566 18022 consensus_queue.cc:237] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } }
I20260811 11:14:23.641016 18024 sys_catalog.cc:455] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b33456f1ea2148e8b21aa3669300148a. Latest consensus state: current_term: 1 leader_uuid: "b33456f1ea2148e8b21aa3669300148a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } } }
I20260811 11:14:23.641117 18023 sys_catalog.cc:455] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b33456f1ea2148e8b21aa3669300148a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b33456f1ea2148e8b21aa3669300148a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 32837 } } }
I20260811 11:14:23.641227 18024 sys_catalog.cc:458] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:23.641233 18023 sys_catalog.cc:458] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:23.642949 18037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:23.643636 18037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:23.645596 18037 catalog_manager.cc:1347] Generated new cluster ID: 9e950c735cdc4c3b8eb364515356f132
I20260811 11:14:23.645648 18037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:23.647588 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.641666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36114 (local address 127.29.145.62:32837)
0811 11:14:23.641913 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.641924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.641960 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:14:23.642115 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.642124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.642211 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.642703 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:23.642832 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.644102 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.645196 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.645767 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.645944 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.646796 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:23.646801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:23.646802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:23.647038 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:23.647083 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:23.647097 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:23.647160 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.647276 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.647366 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260811 11:14:23.656977 18037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:23.657567 18037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:23.670346 18037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: Generated new TSK 0
I20260811 11:14:23.670624 18037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:23.679565 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--builtin_ntp_servers=127.29.145.20:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:23.783883 18044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:23.784157 18044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:23.784184 18044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:23.785616 18044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:23.785673 18044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:23.785705 18044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:23.787243 18044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:23.787585 18044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:23.787998 18044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:23.793846 18049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.793870 18050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.793864 18052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:23.794199 18044 server_base.cc:1034] running on GCE node
I20260811 11:14:23.794535 18044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:23.794896 18044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:23.796070 18044 hybrid_clock.cc:648] HybridClock initialized: now 1786446863796057 us; error 31 us; skew 500 ppm
I20260811 11:14:23.797684 18044 webserver.cc:460] Webserver started at http://127.29.145.1:39179/ using document root <none> and password file <none>
I20260811 11:14:23.798029 18044 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:23.798130 18044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:23.798302 18044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:23.799564 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "5df5cc438d4d4f7a9506ef361b03c871"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "30b5966a94b517d48657bd0a5ab5064a"
server_key_iv: "51a975e7052ef72810aa42ae29a6c9e1"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.799983 18044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "5df5cc438d4d4f7a9506ef361b03c871"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "30b5966a94b517d48657bd0a5ab5064a"
server_key_iv: "51a975e7052ef72810aa42ae29a6c9e1"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.801465 18044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:23.802330 18059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.802515 18044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:23.802647 18044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "5df5cc438d4d4f7a9506ef361b03c871"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "30b5966a94b517d48657bd0a5ab5064a"
server_key_iv: "51a975e7052ef72810aa42ae29a6c9e1"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.802728 18044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:23.817549 18044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:23.818043 18044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:23.818264 18044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:23.818573 18044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:23.819000 18044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:23.819060 18044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.819105 18044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:23.819134 18044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.826177 18044 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:46623
I20260811 11:14:23.826236 18189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:46623 every 8 connection(s)
I20260811 11:14:23.826720 18044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:14:23.828207 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18044
I20260811 11:14:23.828331 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:23.828593 30276 external_mini_cluster.cc:1428] Setting key 1a9fbc40be9f3dfeac7d9720709f2c60
I20260811 11:14:23.829805 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--builtin_ntp_servers=127.29.145.20:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:14:23.833455 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.827711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:52717 (local address 127.29.145.62:32837)
0811 11:14:23.827785 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.827787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.828105 (+ 318us) server_negotiation.cc:407] Connection header received
0811 11:14:23.828851 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.828852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.828872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.828940 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:23.829857 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.830650 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.831739 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.831958 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.832040 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.833137 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:23.833139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:23.833139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:23.833147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:23.833166 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:23.833171 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:23.833228 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.833337 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.833372 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:14:23.833743 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.827700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32837 (local address 127.29.145.1:52717)
0811 11:14:23.827948 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.827987 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.828738 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.829038 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.829044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.829056 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:23.829723 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.829728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.830752 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.830753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.831640 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.831642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.832051 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.832052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.832241 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.832946 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:23.832947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:23.832992 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:23.833032 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:23.833243 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:23.833250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:23.833253 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.833331 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.833465 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":120,"threads_started":1}
I20260811 11:14:23.834333 18190 heartbeater.cc:347] Connected to a master server at 127.29.145.62:32837
I20260811 11:14:23.834442 18190 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:23.834658 18190 heartbeater.cc:511] Master 127.29.145.62:32837 requested a full tablet report, sending...
I20260811 11:14:23.835250 17972 ts_manager.cc:194] Registered new tserver with Master: 5df5cc438d4d4f7a9506ef361b03c871 (127.29.145.1:46623)
I20260811 11:14:23.835929 17972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.1:52717
W20260811 11:14:23.930413 18193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:23.930752 18193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:23.930799 18193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:23.932026 18193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:23.932094 18193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:23.932138 18193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:23.933614 18193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:23.933961 18193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:23.934337 18193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:23.940096 18203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.940089 18201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:23.940092 18199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:23.940527 18193 server_base.cc:1034] running on GCE node
I20260811 11:14:23.940788 18193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:23.941130 18193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:23.942324 18193 hybrid_clock.cc:648] HybridClock initialized: now 1786446863942301 us; error 48 us; skew 500 ppm
I20260811 11:14:23.943840 18193 webserver.cc:460] Webserver started at http://127.29.145.2:38593/ using document root <none> and password file <none>
I20260811 11:14:23.944168 18193 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:23.944222 18193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:23.944371 18193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:23.945611 18193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "7354dd7835fb4af29a69fe8d86b2d11e"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "5c7973dc098523f07e1d8a4c4d0f5d23"
server_key_iv: "76a4efb0302f62ebb201e3644941b4b7"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.946089 18193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "7354dd7835fb4af29a69fe8d86b2d11e"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "5c7973dc098523f07e1d8a4c4d0f5d23"
server_key_iv: "76a4efb0302f62ebb201e3644941b4b7"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.947631 18193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:23.948493 18210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.948771 18193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:23.948882 18193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "7354dd7835fb4af29a69fe8d86b2d11e"
format_stamp: "Formatted at 2026-08-11 11:14:23 on dist-test-slave-69rg"
server_key: "5c7973dc098523f07e1d8a4c4d0f5d23"
server_key_iv: "76a4efb0302f62ebb201e3644941b4b7"
server_key_version: "encryptionkey@0"
I20260811 11:14:23.948949 18193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:23.959317 18193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:23.959964 18193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:23.960129 18193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:23.960451 18193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:23.960860 18193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:23.960903 18193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.960932 18193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:23.960947 18193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:23.968096 18193 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36931
I20260811 11:14:23.968300 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36931 every 8 connection(s)
I20260811 11:14:23.968621 18193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:23.975171 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.969645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:49793 (local address 127.29.145.62:32837)
0811 11:14:23.969749 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:23.969753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:23.970084 (+ 331us) server_negotiation.cc:407] Connection header received
0811 11:14:23.970703 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:23.970706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:23.970720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:23.970791 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:23.971640 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.972252 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.973383 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.973614 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.973675 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.974800 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:23.974802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:23.974802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:23.974808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:23.974833 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:23.974838 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:23.974912 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:23.975016 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 11:14:23.975070 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:14:23.975441 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:23.969582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32837 (local address 127.29.145.2:49793)
0811 11:14:23.969877 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:23.969928 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:23.970563 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:23.970909 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:23.970915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:23.970929 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:23.971538 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.971543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.972379 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.972380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.973289 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:23.973291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:23.973696 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:23.973697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:23.973892 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:23.974599 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:23.974600 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:23.974636 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:23.974663 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:23.974910 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:23.974918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:23.974921 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:23.975018 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 11:14:23.975142 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
I20260811 11:14:23.976086 18345 heartbeater.cc:347] Connected to a master server at 127.29.145.62:32837
I20260811 11:14:23.976210 18345 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:23.976441 18345 heartbeater.cc:511] Master 127.29.145.62:32837 requested a full tablet report, sending...
I20260811 11:14:23.976922 17972 ts_manager.cc:194] Registered new tserver with Master: 7354dd7835fb4af29a69fe8d86b2d11e (127.29.145.2:36931)
I20260811 11:14:23.977366 17972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.2:49793
I20260811 11:14:23.978679 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18193
I20260811 11:14:23.978776 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:23.978958 30276 external_mini_cluster.cc:1428] Setting key 765359f623af09da5437a06667257709
I20260811 11:14:23.980017 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--builtin_ntp_servers=127.29.145.20:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:24.081980 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:24.082238 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:24.082263 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:24.083492 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:24.083534 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:24.083561 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:24.084962 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:32837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:24.085217 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:24.085544 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:24.091488 18354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:24.091473 18357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:24.091478 18355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:24.092003 18349 server_base.cc:1034] running on GCE node
I20260811 11:14:24.092276 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:24.092602 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:24.093786 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786446864093762 us; error 39 us; skew 500 ppm
I20260811 11:14:24.095548 18349 webserver.cc:460] Webserver started at http://127.29.145.3:34789/ using document root <none> and password file <none>
I20260811 11:14:24.095875 18349 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:24.095932 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:24.096091 18349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:24.097386 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "397dab8ef6ab4255838d0f5501a77dc8"
format_stamp: "Formatted at 2026-08-11 11:14:24 on dist-test-slave-69rg"
server_key: "511e23cd25a67229399699155f7b7c86"
server_key_iv: "6983873425bfca090b7c3fb4cbceae88"
server_key_version: "encryptionkey@0"
I20260811 11:14:24.097808 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "397dab8ef6ab4255838d0f5501a77dc8"
format_stamp: "Formatted at 2026-08-11 11:14:24 on dist-test-slave-69rg"
server_key: "511e23cd25a67229399699155f7b7c86"
server_key_iv: "6983873425bfca090b7c3fb4cbceae88"
server_key_version: "encryptionkey@0"
I20260811 11:14:24.099448 18349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:24.100327 18363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:24.100546 18349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:24.100685 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "397dab8ef6ab4255838d0f5501a77dc8"
format_stamp: "Formatted at 2026-08-11 11:14:24 on dist-test-slave-69rg"
server_key: "511e23cd25a67229399699155f7b7c86"
server_key_iv: "6983873425bfca090b7c3fb4cbceae88"
server_key_version: "encryptionkey@0"
I20260811 11:14:24.100771 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:24.124158 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:24.124658 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:24.124868 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:24.125245 18349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:24.125705 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:24.125770 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:24.125872 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:24.125926 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:24.133982 18349 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:36475
I20260811 11:14:24.134079 18503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:36475 every 8 connection(s)
I20260811 11:14:24.134528 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:24.141852 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.135636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59033 (local address 127.29.145.62:32837)
0811 11:14:24.135766 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.135770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.136049 (+ 279us) server_negotiation.cc:407] Connection header received
0811 11:14:24.136866 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.136868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.136885 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.136944 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:24.137870 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.138524 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.139755 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.140061 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.140152 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.141310 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:24.141312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:24.141312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:24.141320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:24.141376 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:24.141382 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:24.141491 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.141640 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.141684 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:14:24.142153 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.135594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:32837 (local address 127.29.145.3:59033)
0811 11:14:24.135893 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.135938 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.136723 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.137037 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.137042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.137054 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:24.137735 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.137740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.138668 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.138670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.139634 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.139637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.140161 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.140163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.140334 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.141088 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:24.141088 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:24.141139 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:24.141177 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:24.141522 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:24.141530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:24.141536 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.141630 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.141779 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
I20260811 11:14:24.142221 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18349
I20260811 11:14:24.142338 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:24.142577 30276 external_mini_cluster.cc:1428] Setting key 7b3409e70f8c580313bcb33f755156ac
I20260811 11:14:24.142822 18504 heartbeater.cc:347] Connected to a master server at 127.29.145.62:32837
I20260811 11:14:24.142947 18504 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:24.143209 18504 heartbeater.cc:511] Master 127.29.145.62:32837 requested a full tablet report, sending...
I20260811 11:14:24.143687 17972 ts_manager.cc:194] Registered new tserver with Master: 397dab8ef6ab4255838d0f5501a77dc8 (127.29.145.3:36475)
I20260811 11:14:24.144192 17972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.3:59033
I20260811 11:14:24.155201 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:24.161401 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.157537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.29.145.62:32837)
0811 11:14:24.157625 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.157628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.157900 (+ 272us) server_negotiation.cc:407] Connection header received
0811 11:14:24.158018 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.158019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.158035 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.158117 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:24.158397 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.158983 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.159947 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.160153 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.160216 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.161071 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:24.161073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:24.161074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:24.161080 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:24.161099 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:24.161104 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:24.161153 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.161220 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.161251 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:14:24.163446 17972 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:24.165635 17972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:24.173796 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.29.145.3:36475)
0811 11:14:24.170149 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.170154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.170171 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:14:24.170230 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.170232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.170247 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.170332 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:24.170457 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171527 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.172628 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.173342 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.173423 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.173524 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.173665 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.173708 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":140,"threads_started":1}
I20260811 11:14:24.173782 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36475 (local address 127.0.0.1:41970)
0811 11:14:24.169588 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.169597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.169637 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.170339 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.170340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.170342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:24.170379 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.170381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171588 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.171589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.172516 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.172519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.173432 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.173434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.173495 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.173507 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.173618 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.173682 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260811 11:14:24.173918 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46623 (local address 127.0.0.1:37822)
0811 11:14:24.169548 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.169558 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.169626 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.170047 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.170051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.170053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:24.170150 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.170155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171538 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.171539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.172758 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.172760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.173645 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.173646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.173725 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.173729 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.173760 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.173800 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":86,"thread_start_us":49,"threads_started":1}
I20260811 11:14:24.174755 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36931 (local address 127.0.0.1:42900)
0811 11:14:24.169305 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.169353 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.169435 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.170102 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.170102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.170104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:24.170150 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.170152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171326 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.171328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.172751 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.172754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.174495 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.174496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.174559 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.174563 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.174638 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.174681 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":45,"thread_start_us":85,"threads_started":1}
I20260811 11:14:24.174783 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.29.145.2:36931)
0811 11:14:24.169913 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.169918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.169929 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:14:24.169976 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.169978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.169994 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.170230 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:24.170318 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171225 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.173580 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.174403 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.174476 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.174558 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.174654 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.174693 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":126,"threads_started":1}
I20260811 11:14:24.174814 18419 tablet_service.cc:1467] Processing CreateTablet for tablet 460ed02c0f804867be9f53c5c625ef2c (DEFAULT_TABLE table=test-table [id=ef9a514133cc445fab74bd6a72af186c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:24.174783 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.169509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.29.145.1:46623)
0811 11:14:24.169768 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.169773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.169804 (+ 31us) server_negotiation.cc:407] Connection header received
0811 11:14:24.169885 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.169887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.169905 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.170029 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:24.170231 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.171431 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.172843 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.173548 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.174530 (+ 982us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.174628 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.174654 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.174693 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260811 11:14:24.175331 18419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460ed02c0f804867be9f53c5c625ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:24.175657 18114 tablet_service.cc:1467] Processing CreateTablet for tablet 460ed02c0f804867be9f53c5c625ef2c (DEFAULT_TABLE table=test-table [id=ef9a514133cc445fab74bd6a72af186c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:24.175985 18114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460ed02c0f804867be9f53c5c625ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:24.178650 18519 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Bootstrap starting.
I20260811 11:14:24.178603 18264 tablet_service.cc:1467] Processing CreateTablet for tablet 460ed02c0f804867be9f53c5c625ef2c (DEFAULT_TABLE table=test-table [id=ef9a514133cc445fab74bd6a72af186c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:24.178977 18264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 460ed02c0f804867be9f53c5c625ef2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:24.179042 18520 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Bootstrap starting.
I20260811 11:14:24.179639 18519 tablet_bootstrap.cc:654] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:24.179684 18520 tablet_bootstrap.cc:654] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:24.180002 18519 log.cc:824] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:24.180002 18520 log.cc:824] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:24.180814 18519 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: No bootstrap required, opened a new log
I20260811 11:14:24.180825 18520 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: No bootstrap required, opened a new log
I20260811 11:14:24.180891 18520 ts_tablet_manager.cc:1397] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:24.180891 18519 ts_tablet_manager.cc:1397] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:24.181864 18523 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Bootstrap starting.
I20260811 11:14:24.182538 18523 tablet_bootstrap.cc:654] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:24.182855 18523 log.cc:824] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:24.183550 18523 tablet_bootstrap.cc:492] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: No bootstrap required, opened a new log
I20260811 11:14:24.183640 18523 ts_tablet_manager.cc:1397] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:24.183552 18519 raft_consensus.cc:348] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.183743 18519 raft_consensus.cc:374] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:24.183795 18519 raft_consensus.cc:729] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397dab8ef6ab4255838d0f5501a77dc8, State: Initialized, Role: FOLLOWER
I20260811 11:14:24.183745 18520 raft_consensus.cc:348] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.183888 18520 raft_consensus.cc:374] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:24.183933 18520 raft_consensus.cc:729] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df5cc438d4d4f7a9506ef361b03c871, State: Initialized, Role: FOLLOWER
I20260811 11:14:24.183952 18519 consensus_queue.cc:260] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.184032 18520 consensus_queue.cc:260] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.184262 18519 ts_tablet_manager.cc:1428] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 11:14:24.184288 18520 ts_tablet_manager.cc:1428] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 11:14:24.184402 18190 heartbeater.cc:503] Master 127.29.145.62:32837 was elected leader, sending a full tablet report...
I20260811 11:14:24.184427 18504 heartbeater.cc:503] Master 127.29.145.62:32837 was elected leader, sending a full tablet report...
I20260811 11:14:24.186461 18523 raft_consensus.cc:348] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.186654 18523 raft_consensus.cc:374] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:24.186702 18523 raft_consensus.cc:729] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7354dd7835fb4af29a69fe8d86b2d11e, State: Initialized, Role: FOLLOWER
I20260811 11:14:24.186816 18523 consensus_queue.cc:260] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.187059 18523 ts_tablet_manager.cc:1428] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 11:14:24.187129 18345 heartbeater.cc:503] Master 127.29.145.62:32837 was elected leader, sending a full tablet report...
W20260811 11:14:24.219281 18347 tablet.cc:2369] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:24.306389 18526 raft_consensus.cc:482] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:24.306576 18526 raft_consensus.cc:504] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.307026 18526 leader_election.cc:288] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7354dd7835fb4af29a69fe8d86b2d11e (127.29.145.2:36931), 397dab8ef6ab4255838d0f5501a77dc8 (127.29.145.3:36475)
I20260811 11:14:24.310235 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.307354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:56167 (local address 127.29.145.2:36931)
0811 11:14:24.307440 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.307445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.307455 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:24.307503 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.307504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.307516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.307601 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:24.307793 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.308365 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.309466 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.309870 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.309939 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.309947 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.310105 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.310159 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:14:24.310251 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.307247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36931 (local address 127.29.145.1:56167)
0811 11:14:24.307328 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.307341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.307390 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.307612 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.307614 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.307616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:24.307700 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.307703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.308472 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.308473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.309365 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.309368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.309952 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.309954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.310027 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.310030 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.310088 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.310122 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":45}
I20260811 11:14:24.310308 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.307542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:44507 (local address 127.29.145.3:36475)
0811 11:14:24.307612 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.307614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.307647 (+ 33us) server_negotiation.cc:407] Connection header received
0811 11:14:24.307679 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.307680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.307690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.307758 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:24.307978 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.308470 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.309477 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.309902 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.309990 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.309998 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.310187 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.310231 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:14:24.310520 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.307402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:36475 (local address 127.29.145.1:44507)
0811 11:14:24.307541 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.307552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:24.307624 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:24.307787 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:24.307788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:24.307790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:24.307871 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.307873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.308557 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.308558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.309363 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:24.309366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.310005 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.310007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:24.310101 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.310105 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:24.310164 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 11:14:24.310200 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":4,"thread_start_us":76,"threads_started":1}
I20260811 11:14:24.310653 18291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460ed02c0f804867be9f53c5c625ef2c" candidate_uuid: "5df5cc438d4d4f7a9506ef361b03c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" is_pre_election: true
I20260811 11:14:24.310837 18291 raft_consensus.cc:2454] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df5cc438d4d4f7a9506ef361b03c871 in term 0.
I20260811 11:14:24.311133 18062 leader_election.cc:302] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df5cc438d4d4f7a9506ef361b03c871, 7354dd7835fb4af29a69fe8d86b2d11e; no voters:
I20260811 11:14:24.311196 18445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460ed02c0f804867be9f53c5c625ef2c" candidate_uuid: "5df5cc438d4d4f7a9506ef361b03c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397dab8ef6ab4255838d0f5501a77dc8" is_pre_election: true
I20260811 11:14:24.311290 18526 raft_consensus.cc:2781] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:24.311358 18526 raft_consensus.cc:482] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:24.311386 18526 raft_consensus.cc:3037] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:24.311420 18445 raft_consensus.cc:2454] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df5cc438d4d4f7a9506ef361b03c871 in term 0.
I20260811 11:14:24.312186 18526 raft_consensus.cc:504] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.312333 18526 leader_election.cc:288] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [CANDIDATE]: Term 1 election: Requested vote from peers 7354dd7835fb4af29a69fe8d86b2d11e (127.29.145.2:36931), 397dab8ef6ab4255838d0f5501a77dc8 (127.29.145.3:36475)
I20260811 11:14:24.312520 18291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460ed02c0f804867be9f53c5c625ef2c" candidate_uuid: "5df5cc438d4d4f7a9506ef361b03c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7354dd7835fb4af29a69fe8d86b2d11e"
I20260811 11:14:24.312553 18445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "460ed02c0f804867be9f53c5c625ef2c" candidate_uuid: "5df5cc438d4d4f7a9506ef361b03c871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "397dab8ef6ab4255838d0f5501a77dc8"
I20260811 11:14:24.312618 18291 raft_consensus.cc:3037] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:24.312623 18445 raft_consensus.cc:3037] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:24.313366 18291 raft_consensus.cc:2454] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df5cc438d4d4f7a9506ef361b03c871 in term 1.
I20260811 11:14:24.313385 18445 raft_consensus.cc:2454] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df5cc438d4d4f7a9506ef361b03c871 in term 1.
I20260811 11:14:24.313566 18062 leader_election.cc:302] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 397dab8ef6ab4255838d0f5501a77dc8, 5df5cc438d4d4f7a9506ef361b03c871; no voters:
I20260811 11:14:24.313684 18526 raft_consensus.cc:2781] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:24.313902 18526 raft_consensus.cc:686] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [term 1 LEADER]: Becoming Leader. State: Replica: 5df5cc438d4d4f7a9506ef361b03c871, State: Running, Role: LEADER
I20260811 11:14:24.314018 18526 consensus_queue.cc:237] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } }
I20260811 11:14:24.314947 17972 catalog_manager.cc:5495] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5df5cc438d4d4f7a9506ef361b03c871 (127.29.145.1). New cstate: current_term: 1 leader_uuid: "5df5cc438d4d4f7a9506ef361b03c871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df5cc438d4d4f7a9506ef361b03c871" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 46623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 } health_report { overall_health: UNKNOWN } } }
W20260811 11:14:24.327577 18191 tablet.cc:2369] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:24.357070 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.354507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.29.145.1:46623)
0811 11:14:24.354571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:24.354573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:24.354589 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:14:24.354633 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:24.354634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:24.354655 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:24.354725 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:24.354864 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.355349 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.356180 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:24.356444 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:24.356502 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:24.356721 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:24.356839 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:24.356917 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:24.356964 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 11:14:24.356986 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:14:24.359233 18291 raft_consensus.cc:1261] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 1 FOLLOWER]: Refusing update from remote peer 5df5cc438d4d4f7a9506ef361b03c871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:24.359614 18445 raft_consensus.cc:1261] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 1 FOLLOWER]: Refusing update from remote peer 5df5cc438d4d4f7a9506ef361b03c871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:24.359642 18526 consensus_queue.cc:1035] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7354dd7835fb4af29a69fe8d86b2d11e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:24.359988 18526 consensus_queue.cc:1035] T 460ed02c0f804867be9f53c5c625ef2c P 5df5cc438d4d4f7a9506ef361b03c871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "397dab8ef6ab4255838d0f5501a77dc8" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 36475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:24.362756 18534 mvcc.cc:205] Tried to move back new op lower bound from 7317286356412370944 to 7317286356230848512. Current Snapshot: MvccSnapshot[applied={T|T < 7317286356412370944}]
I20260811 11:14:24.365037 17972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:24.365140 17972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:24.365916 17972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b33456f1ea2148e8b21aa3669300148a: Sending DeleteTablet for 3 replicas of tablet 460ed02c0f804867be9f53c5c625ef2c
I20260811 11:14:24.366331 18264 tablet_service.cc:1514] Processing DeleteTablet for tablet 460ed02c0f804867be9f53c5c625ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:24 UTC) from {username='slave'} at 127.0.0.1:42900
I20260811 11:14:24.366324 18114 tablet_service.cc:1514] Processing DeleteTablet for tablet 460ed02c0f804867be9f53c5c625ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:24 UTC) from {username='slave'} at 127.0.0.1:37822
I20260811 11:14:24.366596 18546 tablet_replica.cc:330] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: stopping tablet replica
I20260811 11:14:24.366616 18419 tablet_service.cc:1514] Processing DeleteTablet for tablet 460ed02c0f804867be9f53c5c625ef2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:24 UTC) from {username='slave'} at 127.0.0.1:41970
I20260811 11:14:24.366878 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18044
I20260811 11:14:24.366806 18546 raft_consensus.cc:2229] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:24.370225 18546 raft_consensus.cc:2258] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:24.370644 18548 tablet_replica.cc:330] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: stopping tablet replica
I20260811 11:14:24.370761 18548 raft_consensus.cc:2229] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:24.370852 18548 raft_consensus.cc:2258] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:24.370914 18546 ts_tablet_manager.cc:1905] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:24.371280 18548 ts_tablet_manager.cc:1905] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:24.372898 18548 ts_tablet_manager.cc:1918] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:24.372954 18548 log.cc:1196] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/460ed02c0f804867be9f53c5c625ef2c
I20260811 11:14:24.372939 18546 ts_tablet_manager.cc:1918] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:24.373019 18546 log.cc:1196] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/460ed02c0f804867be9f53c5c625ef2c
I20260811 11:14:24.373142 18548 ts_tablet_manager.cc:1939] T 460ed02c0f804867be9f53c5c625ef2c P 397dab8ef6ab4255838d0f5501a77dc8: Deleting consensus metadata
I20260811 11:14:24.373299 18546 ts_tablet_manager.cc:1939] T 460ed02c0f804867be9f53c5c625ef2c P 7354dd7835fb4af29a69fe8d86b2d11e: Deleting consensus metadata
I20260811 11:14:24.373545 17957 catalog_manager.cc:4841] TS 397dab8ef6ab4255838d0f5501a77dc8 (127.29.145.3:36475): tablet 460ed02c0f804867be9f53c5c625ef2c (table test-table [id=ef9a514133cc445fab74bd6a72af186c]) successfully deleted
I20260811 11:14:24.373792 17957 catalog_manager.cc:4841] TS 7354dd7835fb4af29a69fe8d86b2d11e (127.29.145.2:36931): tablet 460ed02c0f804867be9f53c5c625ef2c (table test-table [id=ef9a514133cc445fab74bd6a72af186c]) successfully deleted
W20260811 11:14:24.377907 17957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:46623 (error 108)
I20260811 11:14:24.378252 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18193
I20260811 11:14:24.380527 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:24.378350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:46623 (local address 127.0.0.1:37844)
0811 11:14:24.380437 (+ 2087us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:24.380486 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2062}
W20260811 11:14:24.380720 17957 catalog_manager.cc:4584] TS 5df5cc438d4d4f7a9506ef361b03c871 (127.29.145.1:46623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 460ed02c0f804867be9f53c5c625ef2c: Network error: Client connection negotiation failed: client connection to 127.29.145.1:46623: connect: Connection refused (error 111)
I20260811 11:14:24.388283 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18349
I20260811 11:14:24.395056 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 17936
2026-08-11T11:14:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (934 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:24 dist-test-slave-69rg krb5kdc[18553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:24 dist-test-slave-69rg krb5kdc[18553](info): set up 2 sockets
Aug 11 11:14:24 dist-test-slave-69rg krb5kdc[18553](info): commencing operation
krb5kdc: starting...
W20260811 11:14:25.563367 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:25 dist-test-slave-69rg krb5kdc[18553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:25.694630 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:36349
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:25.783587 18570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:25.783788 18570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:25.783810 18570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:25.784936 18570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:25.784979 18570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:25.784994 18570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:25.785006 18570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:25.785018 18570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:25.787010 18570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:36349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:36349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:25.787315 18570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:25.787581 18570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:25.793170 18578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:25.793206 18575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:25.793208 18576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:25.793268 18570 server_base.cc:1034] running on GCE node
I20260811 11:14:25.793857 18570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:25.794190 18570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:25.795383 18570 hybrid_clock.cc:648] HybridClock initialized: now 1786446865795368 us; error 46 us; skew 500 ppm
Aug 11 11:14:25 dist-test-slave-69rg krb5kdc[18553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446865, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:25.798132 18570 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:25.798630 18570 webserver.cc:460] Webserver started at http://127.29.145.62:37809/ using document root <none> and password file <none>
I20260811 11:14:25.798890 18570 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:25.798961 18570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:25.799109 18570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:25.800179 18570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "b292dc7616d148359ae1c484b00f5653"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "05498084b3992fa8a51a99ed9284415b"
server_key_iv: "39b7b60a4193bf1c8a6a4205ef902a06"
server_key_version: "encryptionkey@0"
I20260811 11:14:25.800549 18570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "b292dc7616d148359ae1c484b00f5653"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "05498084b3992fa8a51a99ed9284415b"
server_key_iv: "39b7b60a4193bf1c8a6a4205ef902a06"
server_key_version: "encryptionkey@0"
I20260811 11:14:25.801998 18570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:25.802779 18587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:25.802968 18570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:25.803069 18570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "b292dc7616d148359ae1c484b00f5653"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "05498084b3992fa8a51a99ed9284415b"
server_key_iv: "39b7b60a4193bf1c8a6a4205ef902a06"
server_key_version: "encryptionkey@0"
I20260811 11:14:25.803154 18570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:25.826817 18570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:25.829249 18570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:25.829418 18570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:25.834044 18570 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:36349
I20260811 11:14:25.834086 18658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:36349 every 8 connection(s)
I20260811 11:14:25.834457 18570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:25.835338 18659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:25.837893 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: Bootstrap starting.
I20260811 11:14:25.838543 18659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:25.838841 18659 log.cc:824] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:25.839586 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: No bootstrap required, opened a new log
I20260811 11:14:25.840250 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18570
I20260811 11:14:25.840374 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:25.840581 30276 external_mini_cluster.cc:1428] Setting key 2f63aaae99b305828f30b3c7b8ae6b71
I20260811 11:14:25.841897 18659 raft_consensus.cc:348] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } }
I20260811 11:14:25.842078 18659 raft_consensus.cc:374] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:25.842119 18659 raft_consensus.cc:729] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b292dc7616d148359ae1c484b00f5653, State: Initialized, Role: FOLLOWER
I20260811 11:14:25.842274 18659 consensus_queue.cc:260] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } }
I20260811 11:14:25.842356 18659 raft_consensus.cc:388] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:25.842379 18659 raft_consensus.cc:482] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:25.842428 18659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:25.843631 18659 raft_consensus.cc:504] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } }
I20260811 11:14:25.843742 18659 leader_election.cc:302] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b292dc7616d148359ae1c484b00f5653; no voters:
I20260811 11:14:25.843881 18659 leader_election.cc:288] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:25.844041 18666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:25.844118 18659 sys_catalog.cc:564] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:25.844205 18666 raft_consensus.cc:686] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [term 1 LEADER]: Becoming Leader. State: Replica: b292dc7616d148359ae1c484b00f5653, State: Running, Role: LEADER
I20260811 11:14:25.844345 18666 consensus_queue.cc:237] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } }
I20260811 11:14:25.844743 18666 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b292dc7616d148359ae1c484b00f5653. Latest consensus state: current_term: 1 leader_uuid: "b292dc7616d148359ae1c484b00f5653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } } }
I20260811 11:14:25.844893 18666 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:25.845788 18667 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b292dc7616d148359ae1c484b00f5653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292dc7616d148359ae1c484b00f5653" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 36349 } } }
I20260811 11:14:25.845956 18667 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653 [sys.catalog]: This master's current role is: LEADER
W20260811 11:14:25.846007 18682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:25.846057 18682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:25.846590 18675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:25.846808 18675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 11:14:25 dist-test-slave-69rg krb5kdc[18553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:25.849359 18675 catalog_manager.cc:1347] Generated new cluster ID: 6c0d4a28a18f478f9ce34cffecab15a1
I20260811 11:14:25.849411 18675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:25.853351 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:25.841180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.29.145.62:36349)
0811 11:14:25.841396 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:25.841409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:25.841447 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:14:25.841844 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:25.841854 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:25.841918 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:25.842062 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:25.842892 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:25.844333 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:25.845469 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:25.846137 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:25.846245 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:25.849451 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:25.849457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:25.849457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:25.849470 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:25.851552 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:25.851963 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:25.851966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:25.851967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:25.852131 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:25.852351 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:25.852352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:25.852353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:25.852710 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:25.852876 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:25.852976 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 11:14:25.853140 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260811 11:14:25.858426 18675 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:25.858899 18675 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:25.862294 18675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: Generated new TSK 0
I20260811 11:14:25.862460 18675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:14:25.893427 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:25.983402 18689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:25.983639 18689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:25.983678 18689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:25.984812 18689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:25.984876 18689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:25.984921 18689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:25.986459 18689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:25.986738 18689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:25.986975 18689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:25.992211 18697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:25.992211 18694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:25.992229 18695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:25.992429 18689 server_base.cc:1034] running on GCE node
I20260811 11:14:25.992722 18689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:25.992967 18689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:25.994125 18689 hybrid_clock.cc:648] HybridClock initialized: now 1786446865994122 us; error 36 us; skew 500 ppm
Aug 11 11:14:25 dist-test-slave-69rg krb5kdc[18553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446865, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:25.996626 18689 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:14:25.997108 18689 webserver.cc:460] Webserver started at http://127.29.145.1:34661/ using document root <none> and password file <none>
I20260811 11:14:25.997360 18689 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:25.997429 18689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:25.997579 18689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:25.998723 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "c5aa937a70a442c09acb77f71a17d21c"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "fe53d4345a3abb6a82294b661617e1eb"
server_key_iv: "7c66a9d7022349a8d6c72320fec50e8a"
server_key_version: "encryptionkey@0"
I20260811 11:14:25.999112 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "c5aa937a70a442c09acb77f71a17d21c"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "fe53d4345a3abb6a82294b661617e1eb"
server_key_iv: "7c66a9d7022349a8d6c72320fec50e8a"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.000442 18689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:14:26.001158 18706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.001339 18689 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:14:26.001446 18689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "c5aa937a70a442c09acb77f71a17d21c"
format_stamp: "Formatted at 2026-08-11 11:14:25 on dist-test-slave-69rg"
server_key: "fe53d4345a3abb6a82294b661617e1eb"
server_key_iv: "7c66a9d7022349a8d6c72320fec50e8a"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.001521 18689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:26.010311 18689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:26.012460 18689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:26.012638 18689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:26.012882 18689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:26.013221 18689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:26.013278 18689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.013316 18689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:26.013343 18689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.019800 18689 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:37905
I20260811 11:14:26.019874 18844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:37905 every 8 connection(s)
I20260811 11:14:26.020238 18689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:14:26 dist-test-slave-69rg krb5kdc[18553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446865, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:26.030006 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.021069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48261 (local address 127.29.145.62:36349)
0811 11:14:26.021195 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.021198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.021372 (+ 174us) server_negotiation.cc:407] Connection header received
0811 11:14:26.021910 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.021911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.021924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.021995 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.022964 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.023506 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.024587 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.024798 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.024858 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.027059 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.027060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.027061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:26.027070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.028591 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.028992 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.028993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.028993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.029047 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.029305 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.029306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.029306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.029410 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.029463 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.029799 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.029938 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:14:26.030115 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.021054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36349 (local address 127.29.145.1:48261)
0811 11:14:26.021234 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.021271 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.021741 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.022083 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.022086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.022654 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:26.022849 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.022852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.023614 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.023615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.024494 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.024496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.024880 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.024881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.024985 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.025510 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:26.025518 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:26.026963 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:26.028703 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.028710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.028711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.028921 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.029140 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.029141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.029141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.029244 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.029502 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:26.029509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:26.029661 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.029758 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.029901 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260811 11:14:26.030349 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18689
I20260811 11:14:26.030458 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:26.030675 30276 external_mini_cluster.cc:1428] Setting key d479fe1e70109140a803614c3c3dcbc1
I20260811 11:14:26.030809 18845 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36349
I20260811 11:14:26.030921 18845 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:26.031134 18845 heartbeater.cc:511] Master 127.29.145.62:36349 requested a full tablet report, sending...
I20260811 11:14:26.031684 18607 ts_manager.cc:194] Registered new tserver with Master: c5aa937a70a442c09acb77f71a17d21c (127.29.145.1:37905)
I20260811 11:14:26.032328 18607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:48261
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:14:26.060726 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:26.148377 18850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:26.148588 18850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:26.148612 18850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:26.149698 18850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:26.149732 18850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:26.149758 18850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:26.151283 18850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:26.151479 18850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:26.151675 18850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:26.156975 18858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.156963 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.157212 18850 server_base.cc:1034] running on GCE node
W20260811 11:14:26.156965 18857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.157558 18850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:26.157781 18850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:26.158968 18850 hybrid_clock.cc:648] HybridClock initialized: now 1786446866158938 us; error 50 us; skew 500 ppm
Aug 11 11:14:26 dist-test-slave-69rg krb5kdc[18553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:26.161679 18850 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:14:26.162199 18850 webserver.cc:460] Webserver started at http://127.29.145.2:39495/ using document root <none> and password file <none>
I20260811 11:14:26.162403 18850 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:26.162446 18850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:26.162536 18850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:26.163565 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "0d7b64ab9a8441ffba37e0ad519db09e"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "aad9b851957e6dd0d9adde0da180d2e7"
server_key_iv: "ec9795986e4f013cc22f70199af92623"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.163897 18850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "0d7b64ab9a8441ffba37e0ad519db09e"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "aad9b851957e6dd0d9adde0da180d2e7"
server_key_iv: "ec9795986e4f013cc22f70199af92623"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.165264 18850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:26.166365 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.166558 18850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:26.166666 18850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "0d7b64ab9a8441ffba37e0ad519db09e"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "aad9b851957e6dd0d9adde0da180d2e7"
server_key_iv: "ec9795986e4f013cc22f70199af92623"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.166733 18850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:26.184746 18850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:26.186905 18850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:26.187027 18850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:26.187227 18850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:26.187574 18850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:26.187611 18850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.187635 18850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:26.187682 18850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.193699 18850 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:37939
I20260811 11:14:26.193784 19001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:37939 every 8 connection(s)
I20260811 11:14:26.194119 18850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:26.198019 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 18850
I20260811 11:14:26.198174 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:26.198423 30276 external_mini_cluster.cc:1428] Setting key 80f3927bbf5447faf387f4278baaf8cd
Aug 11 11:14:26 dist-test-slave-69rg krb5kdc[18553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:26.204294 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.194859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:34499 (local address 127.29.145.62:36349)
0811 11:14:26.194929 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.194932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.195167 (+ 235us) server_negotiation.cc:407] Connection header received
0811 11:14:26.195657 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.195658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.195669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.195721 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.196715 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.197270 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.198376 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.198684 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.198763 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.201283 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.201285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.201285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:26.201294 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.202821 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.203241 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.203242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.203243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.203310 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.203578 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.203579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.203579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.203701 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.203780 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.204061 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.204151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:14:26.204523 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.194837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36349 (local address 127.29.145.2:34499)
0811 11:14:26.195044 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.195081 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.195561 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.195816 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.195820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.196387 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:26.196575 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.196580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.197356 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.197357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.198286 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.198289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.198808 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.198809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.198943 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.199540 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:26.199549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:26.201178 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:26.202927 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.202935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.202937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.203161 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.203389 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.203390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.203391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.203502 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.203769 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:26.203776 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:26.203921 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.204038 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.204258 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260811 11:14:26.205376 19002 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36349
I20260811 11:14:26.205462 19002 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:26.205659 19002 heartbeater.cc:511] Master 127.29.145.62:36349 requested a full tablet report, sending...
I20260811 11:14:26.206135 18607 ts_manager.cc:194] Registered new tserver with Master: 0d7b64ab9a8441ffba37e0ad519db09e (127.29.145.2:37939)
I20260811 11:14:26.206554 18607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:34499
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:14:26.229877 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:26.313724 19010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:26.313990 19010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:26.314026 19010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:26.315138 19010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:26.315207 19010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:26.315251 19010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:26.316717 19010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:36349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:26.316980 19010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:26.317224 19010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:26.322432 19018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.322466 19015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.322404 19016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.322688 19010 server_base.cc:1034] running on GCE node
I20260811 11:14:26.322913 19010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:26.323151 19010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:26.324329 19010 hybrid_clock.cc:648] HybridClock initialized: now 1786446866324323 us; error 38 us; skew 500 ppm
Aug 11 11:14:26 dist-test-slave-69rg krb5kdc[18553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:26.327095 19010 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:14:26.327605 19010 webserver.cc:460] Webserver started at http://127.29.145.3:38943/ using document root <none> and password file <none>
I20260811 11:14:26.327860 19010 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:26.327929 19010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:26.328091 19010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:26.329173 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "18504998b8c440939faab6b4ed1a0f43"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "6f41a4c8109051dae80ede047f7917ba"
server_key_iv: "19b8ad55ef04f849c52784af63f13e1c"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.329557 19010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "18504998b8c440939faab6b4ed1a0f43"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "6f41a4c8109051dae80ede047f7917ba"
server_key_iv: "19b8ad55ef04f849c52784af63f13e1c"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.331280 19010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:26.332021 19026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.332178 19010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:14:26.332289 19010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "18504998b8c440939faab6b4ed1a0f43"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "6f41a4c8109051dae80ede047f7917ba"
server_key_iv: "19b8ad55ef04f849c52784af63f13e1c"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.332379 19010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:26.342650 19010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:26.344734 19010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:26.344923 19010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:26.345206 19010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:26.345571 19010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:26.345641 19010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.345683 19010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:26.345710 19010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.352312 19010 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:41671
I20260811 11:14:26.352371 19159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:41671 every 8 connection(s)
I20260811 11:14:26.352722 19010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:26.355675 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19010
I20260811 11:14:26.355804 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:26.355973 30276 external_mini_cluster.cc:1428] Setting key 456b8ee23aba7bf0c224f42e55533d90
Aug 11 11:14:26 dist-test-slave-69rg krb5kdc[18553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446866, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:26.363291 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.353721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:46161 (local address 127.29.145.62:36349)
0811 11:14:26.353864 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.353866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.354028 (+ 162us) server_negotiation.cc:407] Connection header received
0811 11:14:26.354607 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.354609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.354623 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.354675 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.355723 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.356359 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.357597 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.357883 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.357953 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.360261 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.360263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.360263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:26.360272 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.361732 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.362196 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.362197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.362197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.362264 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.362579 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.362580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.362580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.362708 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.362798 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.363073 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.363199 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 11:14:26.363464 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.353625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:36349 (local address 127.29.145.3:46161)
0811 11:14:26.353882 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.353929 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.354474 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.354745 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.354750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.355406 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:26.355612 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.355616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.356470 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.356471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.357369 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.357372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.357960 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.357962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.358123 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.358672 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:26.358681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:26.360139 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:26.361850 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.361857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.361860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.362114 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.362360 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:26.362361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:26.362361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:26.362512 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:26.362780 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:26.362787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:26.362938 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.363051 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.363222 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":98,"threads_started":1}
I20260811 11:14:26.363955 19160 heartbeater.cc:347] Connected to a master server at 127.29.145.62:36349
I20260811 11:14:26.364061 19160 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:26.364215 19160 heartbeater.cc:511] Master 127.29.145.62:36349 requested a full tablet report, sending...
I20260811 11:14:26.364769 18607 ts_manager.cc:194] Registered new tserver with Master: 18504998b8c440939faab6b4ed1a0f43 (127.29.145.3:41671)
I20260811 11:14:26.365129 18607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:46161
I20260811 11:14:26.367863 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:26.376152 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.369315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.29.145.62:36349)
0811 11:14:26.369391 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.369394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.369568 (+ 174us) server_negotiation.cc:407] Connection header received
0811 11:14:26.369609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.369610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.369619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.369666 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.370351 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.370847 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.371641 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.371833 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.371897 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.373362 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.373364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.373364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:26.373371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.374700 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.375113 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.375114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.375114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.375169 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:26.375459 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:26.375461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:26.375461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:26.375634 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.375731 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.375893 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.376007 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:14:26.378183 18607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:26.379827 18607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:26.386698 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37905 (local address 127.0.0.1:59090)
0811 11:14:26.382786 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.382798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.382857 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.383130 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.383134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.383137 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:26.383212 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.383214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384247 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.384248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.385538 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.385541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386395 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.386396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.386478 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.386491 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.386563 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.386619 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":28,"thread_start_us":106,"threads_started":1}
I20260811 11:14:26.386708 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.29.145.1:37905)
0811 11:14:26.382947 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.382952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.382962 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:26.383004 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.383006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.383019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.383115 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:26.383303 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384147 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.385646 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386303 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.386392 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.386456 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.386589 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.386630 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":96,"threads_started":1}
I20260811 11:14:26.386996 19178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.29.145.2:37939)
0811 11:14:26.383089 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.383093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.383105 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:14:26.383146 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.383148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.383164 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.383249 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:26.383334 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384446 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.385751 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386530 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.386617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.386673 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.386882 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.386926 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260811 11:14:26.387033 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:41671 (local address 127.0.0.1:32776)
0811 11:14:26.382678 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.382722 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.382799 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.383706 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.383707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.383709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:26.383757 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.383758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384861 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.384862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.385907 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.385910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386753 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.386754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.386880 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.386883 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.386914 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.386944 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260811 11:14:26.387282 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.29.145.3:41671)
0811 11:14:26.383536 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.383540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.383550 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:26.383586 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.383588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.383603 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.383683 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:26.383821 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384760 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.385994 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386662 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.387026 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.387105 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.387132 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.387175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":981,"thread_start_us":117,"threads_started":1}
I20260811 11:14:26.386981 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.382638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:37939 (local address 127.0.0.1:35958)
0811 11:14:26.382786 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.382796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.382842 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.383245 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.383246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.383248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:26.383286 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.383287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.384565 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.384566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.385652 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.385654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.386714 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.386716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.386792 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.386797 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.386855 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.386900 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260811 11:14:26.387892 19077 tablet_service.cc:1467] Processing CreateTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 (DEFAULT_TABLE table=test-table [id=58b050693bb94abca29db43679d25187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:26.387916 18762 tablet_service.cc:1467] Processing CreateTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 (DEFAULT_TABLE table=test-table [id=58b050693bb94abca29db43679d25187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:26.388203 19077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397971c26d9f43a8a28fbc48f5c369b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:26.388319 18762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397971c26d9f43a8a28fbc48f5c369b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:26.391119 19179 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Bootstrap starting.
I20260811 11:14:26.391140 19180 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Bootstrap starting.
I20260811 11:14:26.391762 19179 tablet_bootstrap.cc:654] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:26.391662 18918 tablet_service.cc:1467] Processing CreateTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 (DEFAULT_TABLE table=test-table [id=58b050693bb94abca29db43679d25187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:26.391829 19180 tablet_bootstrap.cc:654] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:26.392028 18918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 397971c26d9f43a8a28fbc48f5c369b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:26.392328 19179 log.cc:824] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:26.392380 19180 log.cc:824] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:26.393074 19179 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: No bootstrap required, opened a new log
I20260811 11:14:26.393163 19179 ts_tablet_manager.cc:1397] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:26.394515 19180 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: No bootstrap required, opened a new log
I20260811 11:14:26.394591 19180 ts_tablet_manager.cc:1397] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:14:26.395000 19183 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Bootstrap starting.
I20260811 11:14:26.395255 19179 raft_consensus.cc:348] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.395449 19179 raft_consensus.cc:374] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:26.395489 19179 raft_consensus.cc:729] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5aa937a70a442c09acb77f71a17d21c, State: Initialized, Role: FOLLOWER
I20260811 11:14:26.395630 19179 consensus_queue.cc:260] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.395694 19183 tablet_bootstrap.cc:654] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:26.395915 19179 ts_tablet_manager.cc:1428] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:14:26.396052 18845 heartbeater.cc:503] Master 127.29.145.62:36349 was elected leader, sending a full tablet report...
I20260811 11:14:26.396302 19183 log.cc:824] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:26.396601 19180 raft_consensus.cc:348] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.396798 19180 raft_consensus.cc:374] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:26.396848 19180 raft_consensus.cc:729] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18504998b8c440939faab6b4ed1a0f43, State: Initialized, Role: FOLLOWER
I20260811 11:14:26.396970 19180 consensus_queue.cc:260] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.397253 19180 ts_tablet_manager.cc:1428] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:14:26.397281 19160 heartbeater.cc:503] Master 127.29.145.62:36349 was elected leader, sending a full tablet report...
I20260811 11:14:26.397289 19183 tablet_bootstrap.cc:492] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: No bootstrap required, opened a new log
I20260811 11:14:26.397401 19183 ts_tablet_manager.cc:1397] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:26.399044 19183 raft_consensus.cc:348] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.399189 19183 raft_consensus.cc:374] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:26.399235 19183 raft_consensus.cc:729] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d7b64ab9a8441ffba37e0ad519db09e, State: Initialized, Role: FOLLOWER
I20260811 11:14:26.399353 19183 consensus_queue.cc:260] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.399649 19183 ts_tablet_manager.cc:1428] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:26.399670 19002 heartbeater.cc:503] Master 127.29.145.62:36349 was elected leader, sending a full tablet report...
W20260811 11:14:26.444702 19003 tablet.cc:2369] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:26.518035 19187 raft_consensus.cc:482] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:26.518237 19187 raft_consensus.cc:504] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.518608 19187 leader_election.cc:288] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18504998b8c440939faab6b4ed1a0f43 (127.29.145.3:41671), c5aa937a70a442c09acb77f71a17d21c (127.29.145.1:37905)
W20260811 11:14:26.521027 18846 tablet.cc:2369] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:26.522692 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.518826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:41671 (local address 127.29.145.2:35313)
0811 11:14:26.518917 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.518930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.518977 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.519237 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.519238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.519240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:26.519344 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.519346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.520404 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.520406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.521640 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.521644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.522384 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.522385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.522473 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.522476 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.522533 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.522568 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":93}
I20260811 11:14:26.522704 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.518986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:35313 (local address 127.29.145.3:41671)
0811 11:14:26.519040 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.519043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.519056 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:14:26.519092 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.519093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.519106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.519198 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:26.519441 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.520278 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.521743 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.522286 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.522356 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.522364 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.522562 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.522614 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:14:26.522768 19176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.518989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:40263 (local address 127.29.145.1:37905)
0811 11:14:26.519047 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.519050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.519136 (+ 86us) server_negotiation.cc:407] Connection header received
0811 11:14:26.519166 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.519167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.519178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.519241 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:26.519440 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.520309 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.521881 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.522347 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.522433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.522443 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.522622 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.522671 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:14:26.522778 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.518826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37905 (local address 127.29.145.2:40263)
0811 11:14:26.519040 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.519049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.519107 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.519251 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.519251 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.519252 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:26.519346 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.519347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.520404 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.520406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.521656 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.521660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.522453 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.522454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.522510 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.522533 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.522596 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.522635 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":190,"thread_start_us":134,"threads_started":1}
I20260811 11:14:26.523069 19102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397971c26d9f43a8a28fbc48f5c369b2" candidate_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18504998b8c440939faab6b4ed1a0f43" is_pre_election: true
I20260811 11:14:26.523191 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397971c26d9f43a8a28fbc48f5c369b2" candidate_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5aa937a70a442c09acb77f71a17d21c" is_pre_election: true
I20260811 11:14:26.523248 19102 raft_consensus.cc:2454] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7b64ab9a8441ffba37e0ad519db09e in term 0.
I20260811 11:14:26.523367 18786 raft_consensus.cc:2454] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d7b64ab9a8441ffba37e0ad519db09e in term 0.
I20260811 11:14:26.523514 18868 leader_election.cc:302] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7b64ab9a8441ffba37e0ad519db09e, 18504998b8c440939faab6b4ed1a0f43; no voters:
I20260811 11:14:26.523654 19187 raft_consensus.cc:2781] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:26.523721 19187 raft_consensus.cc:482] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:26.523749 19187 raft_consensus.cc:3037] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:26.524487 19187 raft_consensus.cc:504] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.524645 19187 leader_election.cc:288] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [CANDIDATE]: Term 1 election: Requested vote from peers 18504998b8c440939faab6b4ed1a0f43 (127.29.145.3:41671), c5aa937a70a442c09acb77f71a17d21c (127.29.145.1:37905)
I20260811 11:14:26.524825 19102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397971c26d9f43a8a28fbc48f5c369b2" candidate_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18504998b8c440939faab6b4ed1a0f43"
I20260811 11:14:26.524857 18786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "397971c26d9f43a8a28fbc48f5c369b2" candidate_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5aa937a70a442c09acb77f71a17d21c"
I20260811 11:14:26.524914 19102 raft_consensus.cc:3037] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:26.524925 18786 raft_consensus.cc:3037] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:26.525849 18786 raft_consensus.cc:2454] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7b64ab9a8441ffba37e0ad519db09e in term 1.
I20260811 11:14:26.525849 19102 raft_consensus.cc:2454] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d7b64ab9a8441ffba37e0ad519db09e in term 1.
I20260811 11:14:26.526046 18871 leader_election.cc:302] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d7b64ab9a8441ffba37e0ad519db09e, c5aa937a70a442c09acb77f71a17d21c; no voters:
I20260811 11:14:26.526145 19187 raft_consensus.cc:2781] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:26.526307 19187 raft_consensus.cc:686] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 1 LEADER]: Becoming Leader. State: Replica: 0d7b64ab9a8441ffba37e0ad519db09e, State: Running, Role: LEADER
I20260811 11:14:26.526412 19187 consensus_queue.cc:237] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } }
I20260811 11:14:26.527275 18607 catalog_manager.cc:5495] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d7b64ab9a8441ffba37e0ad519db09e (127.29.145.2). New cstate: current_term: 1 leader_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d7b64ab9a8441ffba37e0ad519db09e" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 37939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 } health_report { overall_health: UNKNOWN } } }
I20260811 11:14:26.533953 19178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.531245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.29.145.2:37939)
0811 11:14:26.531308 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.531310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.531329 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:14:26.531409 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.531410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.531426 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.531498 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:26.531706 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.532214 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.533000 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.533223 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.533283 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.533507 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:26.533603 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:26.533697 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.533786 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.533795 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:14:26.535776 18786 raft_consensus.cc:1261] T 397971c26d9f43a8a28fbc48f5c369b2 P c5aa937a70a442c09acb77f71a17d21c [term 1 FOLLOWER]: Refusing update from remote peer 0d7b64ab9a8441ffba37e0ad519db09e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:26.535789 19102 raft_consensus.cc:1261] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 1 FOLLOWER]: Refusing update from remote peer 0d7b64ab9a8441ffba37e0ad519db09e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:26.536150 19187 consensus_queue.cc:1035] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18504998b8c440939faab6b4ed1a0f43" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:26.536278 19190 consensus_queue.cc:1035] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5aa937a70a442c09acb77f71a17d21c" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 37905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:26.538687 19193 mvcc.cc:205] Tried to move back new op lower bound from 7317286365327405056 to 7317286365292756992. Current Snapshot: MvccSnapshot[applied={T|T < 7317286365327405056}]
I20260811 11:14:26.539623 19194 mvcc.cc:205] Tried to move back new op lower bound from 7317286365327405056 to 7317286365292756992. Current Snapshot: MvccSnapshot[applied={T|T < 7317286365327405056}]
I20260811 11:14:26.541409 18606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:26.541515 18606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:26.542493 18606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b292dc7616d148359ae1c484b00f5653: Sending DeleteTablet for 3 replicas of tablet 397971c26d9f43a8a28fbc48f5c369b2
I20260811 11:14:26.542891 18918 tablet_service.cc:1514] Processing DeleteTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:26 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:35958
I20260811 11:14:26.542928 19077 tablet_service.cc:1514] Processing DeleteTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:26 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:32776
I20260811 11:14:26.543169 18762 tablet_service.cc:1514] Processing DeleteTablet for tablet 397971c26d9f43a8a28fbc48f5c369b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:26 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:59090
I20260811 11:14:26.543246 19204 tablet_replica.cc:330] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: stopping tablet replica
I20260811 11:14:26.543345 19204 raft_consensus.cc:2229] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:14:26.543466 19204 raft_consensus.cc:2258] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:26.543582 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18689
I20260811 11:14:26.543946 19204 ts_tablet_manager.cc:1905] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:26.545670 19205 tablet_replica.cc:330] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: stopping tablet replica
I20260811 11:14:26.545805 19205 raft_consensus.cc:2229] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:26.545960 19205 raft_consensus.cc:2258] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:26.546667 19205 ts_tablet_manager.cc:1905] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:26.547415 19204 ts_tablet_manager.cc:1918] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:26.547513 19204 log.cc:1196] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/397971c26d9f43a8a28fbc48f5c369b2
I20260811 11:14:26.547915 19204 ts_tablet_manager.cc:1939] T 397971c26d9f43a8a28fbc48f5c369b2 P 0d7b64ab9a8441ffba37e0ad519db09e: Deleting consensus metadata
I20260811 11:14:26.548336 19205 ts_tablet_manager.cc:1918] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:26.548403 19205 log.cc:1196] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/397971c26d9f43a8a28fbc48f5c369b2
I20260811 11:14:26.548457 18588 catalog_manager.cc:4841] TS 0d7b64ab9a8441ffba37e0ad519db09e (127.29.145.2:37939): tablet 397971c26d9f43a8a28fbc48f5c369b2 (table test-table [id=58b050693bb94abca29db43679d25187]) successfully deleted
I20260811 11:14:26.548676 19205 ts_tablet_manager.cc:1939] T 397971c26d9f43a8a28fbc48f5c369b2 P 18504998b8c440939faab6b4ed1a0f43: Deleting consensus metadata
I20260811 11:14:26.549198 18588 catalog_manager.cc:4841] TS 18504998b8c440939faab6b4ed1a0f43 (127.29.145.3:41671): tablet 397971c26d9f43a8a28fbc48f5c369b2 (table test-table [id=58b050693bb94abca29db43679d25187]) successfully deleted
W20260811 11:14:26.556398 18591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:37905 (error 108)
I20260811 11:14:26.556571 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18850
I20260811 11:14:26.562148 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.561997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:37905 (local address 127.0.0.1:59092)
0811 11:14:26.562054 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.562096 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:37905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 11:14:26.562341 18591 catalog_manager.cc:4584] TS c5aa937a70a442c09acb77f71a17d21c (127.29.145.1:37905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 397971c26d9f43a8a28fbc48f5c369b2: Network error: Client connection negotiation failed: client connection to 127.29.145.1:37905: connect: Connection refused (error 111)
I20260811 11:14:26.568351 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19010
I20260811 11:14:26.575979 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 18570
2026-08-11T11:14:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T11:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:26Z Disabled control of system clock
I20260811 11:14:26.609946 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39053
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:39053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:26.692312 19212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:26.692531 19212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:26.692552 19212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:26.693601 19212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:26.693635 19212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:26.693648 19212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:26.693661 19212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:26.693673 19212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:26.695066 19212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:39053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:26.695268 19212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:26.695502 19212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:26.700522 19220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.700722 19222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.700791 19212 server_base.cc:1034] running on GCE node
W20260811 11:14:26.700505 19219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.701097 19212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:26.701467 19212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:26.702636 19212 hybrid_clock.cc:648] HybridClock initialized: now 1786446866702622 us; error 32 us; skew 500 ppm
I20260811 11:14:26.703985 19212 webserver.cc:460] Webserver started at http://127.29.145.62:45353/ using document root <none> and password file <none>
I20260811 11:14:26.704268 19212 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:26.704319 19212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:26.704457 19212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:26.705628 19212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "3fc8e24be9634381b6ddfb6ca14fde34"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "0da38fc4cc12e3dd15588b5a42064d77"
server_key_iv: "cd98561427b56995163e3a0ecc533bdc"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.706055 19212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "3fc8e24be9634381b6ddfb6ca14fde34"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "0da38fc4cc12e3dd15588b5a42064d77"
server_key_iv: "cd98561427b56995163e3a0ecc533bdc"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.707516 19212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:26.708382 19229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.708590 19212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:26.708698 19212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "3fc8e24be9634381b6ddfb6ca14fde34"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "0da38fc4cc12e3dd15588b5a42064d77"
server_key_iv: "cd98561427b56995163e3a0ecc533bdc"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.708781 19212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:26.719635 19212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:26.719934 19212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:26.720081 19212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:26.724258 19212 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:39053
I20260811 11:14:26.724292 19296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:39053 every 8 connection(s)
I20260811 11:14:26.724689 19212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:26.725696 19297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:26.728164 19297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: Bootstrap starting.
I20260811 11:14:26.728798 19297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:26.729120 19297 log.cc:824] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:26.729928 19297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: No bootstrap required, opened a new log
I20260811 11:14:26.731585 19297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } }
I20260811 11:14:26.731736 19297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:26.731760 19297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fc8e24be9634381b6ddfb6ca14fde34, State: Initialized, Role: FOLLOWER
I20260811 11:14:26.731926 19297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } }
I20260811 11:14:26.731994 19297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:26.732016 19297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:26.732088 19297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:26.732640 19297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } }
I20260811 11:14:26.732743 19297 leader_election.cc:302] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fc8e24be9634381b6ddfb6ca14fde34; no voters:
I20260811 11:14:26.732942 19297 leader_election.cc:288] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:26.733203 19302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:26.733453 19302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [term 1 LEADER]: Becoming Leader. State: Replica: 3fc8e24be9634381b6ddfb6ca14fde34, State: Running, Role: LEADER
I20260811 11:14:26.733461 19297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:26.733569 19302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } }
I20260811 11:14:26.734424 19303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } } }
I20260811 11:14:26.734519 19303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:26.734709 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19212
I20260811 11:14:26.734830 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:26.735037 30276 external_mini_cluster.cc:1428] Setting key 2789a5eee638c9f73f72a170682c675d
I20260811 11:14:26.735783 19305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fc8e24be9634381b6ddfb6ca14fde34. Latest consensus state: current_term: 1 leader_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fc8e24be9634381b6ddfb6ca14fde34" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39053 } } }
I20260811 11:14:26.735872 19305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:26.735999 19313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:26.736485 19313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:26.738148 19313 catalog_manager.cc:1347] Generated new cluster ID: 44f368b056a442f3b837b21d654d8e4a
I20260811 11:14:26.738204 19313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:26.742596 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.735531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.29.145.62:39053)
0811 11:14:26.737103 (+ 1572us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.737111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.737143 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:14:26.737263 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.737270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.737341 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.737504 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.737856 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.739229 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.740325 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.740955 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.741084 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.741843 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.741848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.741848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:26.742071 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.742133 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:26.742148 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.742226 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.742302 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.742382 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1499,"thread_start_us":81,"threads_started":1}
I20260811 11:14:26.752800 19313 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:26.753394 19313 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:26.756132 19313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: Generated new TSK 0
I20260811 11:14:26.756341 19313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:26.763616 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--builtin_ntp_servers=127.29.145.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:26.850517 19330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:26.850725 19330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:26.850746 19330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:26.851881 19330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:26.851923 19330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:26.851948 19330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:26.853291 19330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:26.853549 19330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:26.853920 19330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:26.859212 19335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.859210 19337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:26.859232 19339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:26.859663 19330 server_base.cc:1034] running on GCE node
I20260811 11:14:26.859874 19330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:26.860126 19330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:26.861306 19330 hybrid_clock.cc:648] HybridClock initialized: now 1786446866861281 us; error 40 us; skew 500 ppm
I20260811 11:14:26.862800 19330 webserver.cc:460] Webserver started at http://127.29.145.1:38037/ using document root <none> and password file <none>
I20260811 11:14:26.863096 19330 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:26.863173 19330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:26.863332 19330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:26.864478 19330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "cddf6d0d0645468c966e72d3860dc18a"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "b8e96a55607959355481f1fb4f10d0fd"
server_key_iv: "1c5707e0d1d62bfac6d04106712411c1"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.864856 19330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "cddf6d0d0645468c966e72d3860dc18a"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "b8e96a55607959355481f1fb4f10d0fd"
server_key_iv: "1c5707e0d1d62bfac6d04106712411c1"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.866341 19330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:26.867174 19345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.867368 19330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:26.867480 19330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "cddf6d0d0645468c966e72d3860dc18a"
format_stamp: "Formatted at 2026-08-11 11:14:26 on dist-test-slave-69rg"
server_key: "b8e96a55607959355481f1fb4f10d0fd"
server_key_iv: "1c5707e0d1d62bfac6d04106712411c1"
server_key_version: "encryptionkey@0"
I20260811 11:14:26.867558 19330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:26.900084 19330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:26.900445 19330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:26.900612 19330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:26.900866 19330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:26.901232 19330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:26.901290 19330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.901326 19330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:26.901376 19330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:26.907454 19330 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:33395
I20260811 11:14:26.907506 19484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:33395 every 8 connection(s)
I20260811 11:14:26.907840 19330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:14:26.910601 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19330
I20260811 11:14:26.910712 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:26.910892 30276 external_mini_cluster.cc:1428] Setting key 92c3407f4a53731f7eabdbd1653afad7
I20260811 11:14:26.911942 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--builtin_ntp_servers=127.29.145.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:14:26.914170 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.908583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:45371 (local address 127.29.145.62:39053)
0811 11:14:26.908675 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:26.908677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:26.908931 (+ 254us) server_negotiation.cc:407] Connection header received
0811 11:14:26.909498 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:26.909499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:26.909513 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:26.909573 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:26.910317 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.911113 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.912544 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.912810 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.912875 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.913839 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:26.913840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:26.913841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:26.913849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:26.913866 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:26.913871 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:26.913934 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:26.914030 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 11:14:26.914062 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:14:26.914283 19487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:26.908551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39053 (local address 127.29.145.1:45371)
0811 11:14:26.908805 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:26.908846 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:26.909380 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:26.909626 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:26.909630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:26.909638 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:26.910232 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.910237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.911228 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.911229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.912432 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:26.912436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:26.912906 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:26.912907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:26.913069 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:26.913628 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:26.913629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:26.913674 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:26.913707 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:26.913943 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:26.913949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:26.913951 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:14:26.914007 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 11:14:26.914088 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":129,"threads_started":1}
I20260811 11:14:26.914806 19485 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39053
I20260811 11:14:26.914889 19485 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:26.915107 19485 heartbeater.cc:511] Master 127.29.145.62:39053 requested a full tablet report, sending...
I20260811 11:14:26.915622 19252 ts_manager.cc:194] Registered new tserver with Master: cddf6d0d0645468c966e72d3860dc18a (127.29.145.1:33395)
I20260811 11:14:26.916229 19252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.1:45371
W20260811 11:14:26.997087 19490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:26.997334 19490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:26.997370 19490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:26.998520 19490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:26.998585 19490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:26.998625 19490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:26.999974 19490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:27.000252 19490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:27.000541 19490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:27.005864 19501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:27.005878 19499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:27.005873 19496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:27.006107 19490 server_base.cc:1034] running on GCE node
I20260811 11:14:27.006392 19490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:27.006649 19490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:27.007822 19490 hybrid_clock.cc:648] HybridClock initialized: now 1786446867007819 us; error 40 us; skew 500 ppm
I20260811 11:14:27.009124 19490 webserver.cc:460] Webserver started at http://127.29.145.2:36453/ using document root <none> and password file <none>
I20260811 11:14:27.009407 19490 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:27.009483 19490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:27.009635 19490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:27.010840 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "4b339f6a6aa941aeb33447de3c08c244"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "b87b1c208538a7924f9f8ae8bed435db"
server_key_iv: "d80715a402a420149003377bbf77181d"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.011227 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "4b339f6a6aa941aeb33447de3c08c244"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "b87b1c208538a7924f9f8ae8bed435db"
server_key_iv: "d80715a402a420149003377bbf77181d"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.012842 19490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:27.013727 19509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.013948 19490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:27.014055 19490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "4b339f6a6aa941aeb33447de3c08c244"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "b87b1c208538a7924f9f8ae8bed435db"
server_key_iv: "d80715a402a420149003377bbf77181d"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.014142 19490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:27.038936 19490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:27.039338 19490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:27.039484 19490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:27.039739 19490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:27.040175 19490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:27.040238 19490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.040275 19490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:27.040304 19490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.046190 19490 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:46761
I20260811 11:14:27.046242 19645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:46761 every 8 connection(s)
I20260811 11:14:27.046576 19490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:27.049052 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19490
I20260811 11:14:27.049155 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:27.049408 30276 external_mini_cluster.cc:1428] Setting key 9251360aaf128db865b5a0c294fe1ff1
I20260811 11:14:27.050575 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--builtin_ntp_servers=127.29.145.20:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:14:27.052958 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.047358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:37641 (local address 127.29.145.62:39053)
0811 11:14:27.047416 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.047418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.047740 (+ 322us) server_negotiation.cc:407] Connection header received
0811 11:14:27.048232 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.048233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.048246 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.048310 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:27.049143 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.049875 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.051275 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.051463 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.051524 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.052530 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:27.052531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:27.052532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:27.052538 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:27.052557 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:27.052562 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:27.052627 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.052809 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.052843 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:14:27.053098 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.047263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39053 (local address 127.29.145.2:37641)
0811 11:14:27.047561 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.047603 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.048092 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.048409 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.048413 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.048421 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:27.048998 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.049004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.049977 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.049978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.051167 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.051171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.051559 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.051560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.051723 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.052345 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:27.052346 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:27.052388 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:27.052423 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:27.052701 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:27.052707 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:27.052710 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.052785 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.052872 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":188,"threads_started":1}
I20260811 11:14:27.053673 19646 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39053
I20260811 11:14:27.053791 19646 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:27.054001 19646 heartbeater.cc:511] Master 127.29.145.62:39053 requested a full tablet report, sending...
I20260811 11:14:27.054426 19252 ts_manager.cc:194] Registered new tserver with Master: 4b339f6a6aa941aeb33447de3c08c244 (127.29.145.2:46761)
I20260811 11:14:27.054814 19252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.2:37641
W20260811 11:14:27.137672 19649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:27.138024 19649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:27.138049 19649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:27.139317 19649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:27.139357 19649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:27.139382 19649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:27.140734 19649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:27.140974 19649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:27.141299 19649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:27.146600 19658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:27.146595 19655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:27.146756 19649 server_base.cc:1034] running on GCE node
W20260811 11:14:27.146620 19656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:27.147089 19649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:27.147303 19649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:27.148454 19649 hybrid_clock.cc:648] HybridClock initialized: now 1786446867148431 us; error 45 us; skew 500 ppm
I20260811 11:14:27.149782 19649 webserver.cc:460] Webserver started at http://127.29.145.3:39023/ using document root <none> and password file <none>
I20260811 11:14:27.150058 19649 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:27.150106 19649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:27.150286 19649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:27.151438 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "e9951fa0ee8b4013be8438f133555dde"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "f56931ea0dca5a13253a847e33978734"
server_key_iv: "11ce34f0e17e38d521b942be644a1a60"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.151805 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "e9951fa0ee8b4013be8438f133555dde"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "f56931ea0dca5a13253a847e33978734"
server_key_iv: "11ce34f0e17e38d521b942be644a1a60"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.153358 19649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:27.154189 19666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.154368 19649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.154471 19649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "e9951fa0ee8b4013be8438f133555dde"
format_stamp: "Formatted at 2026-08-11 11:14:27 on dist-test-slave-69rg"
server_key: "f56931ea0dca5a13253a847e33978734"
server_key_iv: "11ce34f0e17e38d521b942be644a1a60"
server_key_version: "encryptionkey@0"
I20260811 11:14:27.154556 19649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:27.165621 19649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:27.166024 19649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:27.166193 19649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:27.166437 19649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:27.166872 19649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:27.166932 19649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.166975 19649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:27.167026 19649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:27.173362 19649 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:39937
I20260811 11:14:27.173404 19799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:39937 every 8 connection(s)
I20260811 11:14:27.173790 19649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:27.177445 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19649
I20260811 11:14:27.177551 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:27.177763 30276 external_mini_cluster.cc:1428] Setting key df431bc027e070390f10ae5419bdad1e
I20260811 11:14:27.179769 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.174666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:55081 (local address 127.29.145.62:39053)
0811 11:14:27.174714 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.174718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.174967 (+ 249us) server_negotiation.cc:407] Connection header received
0811 11:14:27.175499 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.175500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.175513 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.175582 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:27.176402 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.176919 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.178238 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.178507 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.178607 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.179453 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:27.179455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:27.179455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:27.179460 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:27.179476 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:27.179480 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:27.179533 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.179657 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.179690 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:14:27.179993 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.174624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:39053 (local address 127.29.145.3:55081)
0811 11:14:27.174827 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.174869 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.175358 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.175721 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.175725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.175734 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:27.176275 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.176280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.177029 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.177030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.178125 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.178129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.178643 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.178644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.178775 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.179272 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:27.179273 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:27.179307 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:27.179330 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:27.179546 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:27.179551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:27.179554 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.179656 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.179761 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260811 11:14:27.180454 19800 heartbeater.cc:347] Connected to a master server at 127.29.145.62:39053
I20260811 11:14:27.180534 19800 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:27.180729 19800 heartbeater.cc:511] Master 127.29.145.62:39053 requested a full tablet report, sending...
I20260811 11:14:27.181092 19252 ts_manager.cc:194] Registered new tserver with Master: e9951fa0ee8b4013be8438f133555dde (127.29.145.3:39937)
I20260811 11:14:27.181514 19252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.3:55081
I20260811 11:14:27.189795 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:27.194345 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.191334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.29.145.62:39053)
0811 11:14:27.191529 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.191531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.191540 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:27.191569 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.191570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.191581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.191637 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:27.191838 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.192296 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.193030 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.193221 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.193278 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.194013 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:27.194015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:27.194015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:27.194019 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:27.194030 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:27.194034 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:27.194104 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.194225 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.194238 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 11:14:27.195892 19252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:27.197441 19252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:27.204384 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.200047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.29.145.2:46761)
0811 11:14:27.200559 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.200563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.200574 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:14:27.200619 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.200621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.200638 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.200713 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:27.200972 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.201675 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.203387 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.203966 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204061 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.204159 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.204249 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.204290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":99,"threads_started":1}
I20260811 11:14:27.204380 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.199935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:33395 (local address 127.0.0.1:35358)
0811 11:14:27.200265 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.200277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.200333 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.200615 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.200618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.200621 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:27.200693 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.200696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.202093 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.202094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.203215 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.203217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.204074 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.204156 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.204160 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.204209 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.204263 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":119,"thread_start_us":91,"threads_started":1}
I20260811 11:14:27.204600 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.200201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.29.145.1:33395)
0811 11:14:27.200411 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.200416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.200426 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:27.200470 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.200471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.200489 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.200778 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:27.200844 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.202011 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.203310 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.203970 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204336 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.204439 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.204464 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.204507 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260811 11:14:27.204614 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.200136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:39937 (local address 127.0.0.1:53668)
0811 11:14:27.200366 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.200377 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.200428 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.200714 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.200714 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.200716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:27.200753 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.200754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.202502 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.202503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.203523 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.203525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.204416 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.204476 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.204480 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.204509 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.204543 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":67,"threads_started":1}
I20260811 11:14:27.204628 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.199935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:46761 (local address 127.0.0.1:41650)
0811 11:14:27.200103 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.200142 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.200208 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.200793 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.200794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.200796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:27.200882 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.200884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.201774 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.201776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.203272 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.203275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.204061 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.204142 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.204156 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.204233 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.204281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260811 11:14:27.205173 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.200329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.29.145.3:39937)
0811 11:14:27.200537 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.200541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.200551 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:27.200586 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.200596 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.200612 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.200824 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:27.200886 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.201995 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.203610 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.204322 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.204940 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.205031 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.205056 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.205105 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":122,"threads_started":1}
I20260811 11:14:27.205384 19405 tablet_service.cc:1467] Processing CreateTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 (DEFAULT_TABLE table=test-table [id=54f2f49405e84bab958a44fec068ec5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:27.205684 19562 tablet_service.cc:1467] Processing CreateTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 (DEFAULT_TABLE table=test-table [id=54f2f49405e84bab958a44fec068ec5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:27.205788 19405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8daec9d82c4a5b8b1ef30dab5fd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:27.205909 19721 tablet_service.cc:1467] Processing CreateTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 (DEFAULT_TABLE table=test-table [id=54f2f49405e84bab958a44fec068ec5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:27.206039 19562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8daec9d82c4a5b8b1ef30dab5fd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:27.206245 19721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae8daec9d82c4a5b8b1ef30dab5fd823. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:27.208624 19818 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Bootstrap starting.
I20260811 11:14:27.209237 19818 tablet_bootstrap.cc:654] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:27.209426 19819 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: Bootstrap starting.
I20260811 11:14:27.209574 19818 log.cc:824] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:27.209697 19820 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Bootstrap starting.
I20260811 11:14:27.210287 19819 tablet_bootstrap.cc:654] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:27.210399 19820 tablet_bootstrap.cc:654] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:27.210551 19818 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: No bootstrap required, opened a new log
I20260811 11:14:27.210628 19818 ts_tablet_manager.cc:1397] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:27.210649 19819 log.cc:824] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:27.210732 19820 log.cc:824] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:27.211408 19820 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: No bootstrap required, opened a new log
I20260811 11:14:27.211493 19820 ts_tablet_manager.cc:1397] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:27.211494 19819 tablet_bootstrap.cc:492] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: No bootstrap required, opened a new log
I20260811 11:14:27.211562 19819 ts_tablet_manager.cc:1397] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:14:27.213397 19818 raft_consensus.cc:348] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.213574 19818 raft_consensus.cc:374] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:27.213611 19818 raft_consensus.cc:729] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b339f6a6aa941aeb33447de3c08c244, State: Initialized, Role: FOLLOWER
I20260811 11:14:27.213526 19820 raft_consensus.cc:348] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.213691 19820 raft_consensus.cc:374] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:27.213737 19820 raft_consensus.cc:729] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9951fa0ee8b4013be8438f133555dde, State: Initialized, Role: FOLLOWER
I20260811 11:14:27.213773 19818 consensus_queue.cc:260] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.213907 19820 consensus_queue.cc:260] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.214084 19818 ts_tablet_manager.cc:1428] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 11:14:27.214210 19646 heartbeater.cc:503] Master 127.29.145.62:39053 was elected leader, sending a full tablet report...
I20260811 11:14:27.214216 19820 ts_tablet_manager.cc:1428] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:14:27.214286 19800 heartbeater.cc:503] Master 127.29.145.62:39053 was elected leader, sending a full tablet report...
I20260811 11:14:27.214120 19819 raft_consensus.cc:348] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.214566 19819 raft_consensus.cc:374] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:27.214623 19819 raft_consensus.cc:729] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cddf6d0d0645468c966e72d3860dc18a, State: Initialized, Role: FOLLOWER
I20260811 11:14:27.214731 19819 consensus_queue.cc:260] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.215008 19819 ts_tablet_manager.cc:1428] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:14:27.215049 19485 heartbeater.cc:503] Master 127.29.145.62:39053 was elected leader, sending a full tablet report...
W20260811 11:14:27.297174 19647 tablet.cc:2369] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:27.337658 19825 raft_consensus.cc:482] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:27.337798 19825 raft_consensus.cc:504] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.338223 19825 leader_election.cc:288] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cddf6d0d0645468c966e72d3860dc18a (127.29.145.1:33395), 4b339f6a6aa941aeb33447de3c08c244 (127.29.145.2:46761)
I20260811 11:14:27.342410 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.338562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:59083 (local address 127.29.145.2:46761)
0811 11:14:27.338606 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.338609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.338769 (+ 160us) server_negotiation.cc:407] Connection header received
0811 11:14:27.338834 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.338835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.338854 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.338893 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:27.339077 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.339795 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.341396 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.341995 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.342079 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.342088 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.342291 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.342338 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:14:27.342410 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.338546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:38101 (local address 127.29.145.1:33395)
0811 11:14:27.338631 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.338633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.338646 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:14:27.338685 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.338686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.338697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.338778 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:27.338976 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.339754 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.341397 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.341995 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.342079 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.342088 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.342291 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.342338 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:14:27.342413 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.338447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:33395 (local address 127.29.145.3:38101)
0811 11:14:27.338544 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.338557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.338607 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.338795 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.338796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.338798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:27.338895 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.338897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.339899 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.339901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.341268 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.341272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.342096 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.342097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.342206 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.342209 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.342262 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.342317 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":64}
I20260811 11:14:27.342427 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.338502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:46761 (local address 127.29.145.3:59083)
0811 11:14:27.338690 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.338698 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:27.338768 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:27.338920 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:27.338921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:27.338923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:27.338989 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.338992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.339899 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.339920 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.341268 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:27.341271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.342099 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.342099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:27.342190 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.342196 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:14:27.342262 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 11:14:27.342316 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":51,"thread_start_us":99,"threads_started":1}
I20260811 11:14:27.342825 19429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8daec9d82c4a5b8b1ef30dab5fd823" candidate_uuid: "e9951fa0ee8b4013be8438f133555dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cddf6d0d0645468c966e72d3860dc18a" is_pre_election: true
I20260811 11:14:27.342831 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8daec9d82c4a5b8b1ef30dab5fd823" candidate_uuid: "e9951fa0ee8b4013be8438f133555dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b339f6a6aa941aeb33447de3c08c244" is_pre_election: true
I20260811 11:14:27.342988 19429 raft_consensus.cc:2454] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9951fa0ee8b4013be8438f133555dde in term 0.
I20260811 11:14:27.343006 19588 raft_consensus.cc:2454] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9951fa0ee8b4013be8438f133555dde in term 0.
I20260811 11:14:27.343225 19667 leader_election.cc:302] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b339f6a6aa941aeb33447de3c08c244, e9951fa0ee8b4013be8438f133555dde; no voters:
I20260811 11:14:27.343394 19825 raft_consensus.cc:2781] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:27.343456 19825 raft_consensus.cc:482] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:27.343479 19825 raft_consensus.cc:3037] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:27.344185 19825 raft_consensus.cc:504] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.344334 19825 leader_election.cc:288] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [CANDIDATE]: Term 1 election: Requested vote from peers cddf6d0d0645468c966e72d3860dc18a (127.29.145.1:33395), 4b339f6a6aa941aeb33447de3c08c244 (127.29.145.2:46761)
I20260811 11:14:27.344549 19588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8daec9d82c4a5b8b1ef30dab5fd823" candidate_uuid: "e9951fa0ee8b4013be8438f133555dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b339f6a6aa941aeb33447de3c08c244"
I20260811 11:14:27.344549 19429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae8daec9d82c4a5b8b1ef30dab5fd823" candidate_uuid: "e9951fa0ee8b4013be8438f133555dde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cddf6d0d0645468c966e72d3860dc18a"
I20260811 11:14:27.344633 19588 raft_consensus.cc:3037] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:27.344633 19429 raft_consensus.cc:3037] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:27.345543 19429 raft_consensus.cc:2454] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9951fa0ee8b4013be8438f133555dde in term 1.
I20260811 11:14:27.345543 19588 raft_consensus.cc:2454] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9951fa0ee8b4013be8438f133555dde in term 1.
I20260811 11:14:27.345754 19667 leader_election.cc:302] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b339f6a6aa941aeb33447de3c08c244, e9951fa0ee8b4013be8438f133555dde; no voters:
I20260811 11:14:27.345932 19825 raft_consensus.cc:2781] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:27.346149 19825 raft_consensus.cc:686] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 1 LEADER]: Becoming Leader. State: Replica: e9951fa0ee8b4013be8438f133555dde, State: Running, Role: LEADER
I20260811 11:14:27.346268 19825 consensus_queue.cc:237] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } }
I20260811 11:14:27.347252 19250 catalog_manager.cc:5495] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde reported cstate change: term changed from 0 to 1, leader changed from <none> to e9951fa0ee8b4013be8438f133555dde (127.29.145.3). New cstate: current_term: 1 leader_uuid: "e9951fa0ee8b4013be8438f133555dde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9951fa0ee8b4013be8438f133555dde" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260811 11:14:27.354266 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.351820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53672 (local address 127.29.145.3:39937)
0811 11:14:27.351877 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:27.351879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:27.351889 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:27.351956 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:27.351957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:27.351976 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:27.352065 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:27.352205 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.352670 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.353429 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:27.353638 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:27.353717 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:27.353902 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:27.354001 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:27.354115 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:27.354153 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 11:14:27.354161 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 11:14:27.356055 19588 raft_consensus.cc:1261] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 1 FOLLOWER]: Refusing update from remote peer e9951fa0ee8b4013be8438f133555dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:27.356055 19429 raft_consensus.cc:1261] T ae8daec9d82c4a5b8b1ef30dab5fd823 P cddf6d0d0645468c966e72d3860dc18a [term 1 FOLLOWER]: Refusing update from remote peer e9951fa0ee8b4013be8438f133555dde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:27.356395 19825 consensus_queue.cc:1035] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "cddf6d0d0645468c966e72d3860dc18a" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 33395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:27.356570 19828 consensus_queue.cc:1035] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b339f6a6aa941aeb33447de3c08c244" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:27.359022 19832 mvcc.cc:205] Tried to move back new op lower bound from 7317286368687669248 to 7317286368651132928. Current Snapshot: MvccSnapshot[applied={T|T < 7317286368687669248}]
I20260811 11:14:27.359920 19833 mvcc.cc:205] Tried to move back new op lower bound from 7317286368687669248 to 7317286368651132928. Current Snapshot: MvccSnapshot[applied={T|T < 7317286368687669248}]
I20260811 11:14:27.360126 19835 mvcc.cc:205] Tried to move back new op lower bound from 7317286368687669248 to 7317286368651132928. Current Snapshot: MvccSnapshot[applied={T|T < 7317286368687669248}]
I20260811 11:14:27.361647 19250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:27.361745 19250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:27.362622 19250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fc8e24be9634381b6ddfb6ca14fde34: Sending DeleteTablet for 3 replicas of tablet ae8daec9d82c4a5b8b1ef30dab5fd823
I20260811 11:14:27.362946 19405 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:27 UTC) from {username='slave'} at 127.0.0.1:35358
I20260811 11:14:27.362990 19721 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:27 UTC) from {username='slave'} at 127.0.0.1:53668
I20260811 11:14:27.362993 19562 tablet_service.cc:1514] Processing DeleteTablet for tablet ae8daec9d82c4a5b8b1ef30dab5fd823 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:27 UTC) from {username='slave'} at 127.0.0.1:41650
I20260811 11:14:27.363179 19843 tablet_replica.cc:330] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: stopping tablet replica
I20260811 11:14:27.363274 19843 raft_consensus.cc:2229] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:14:27.363373 19843 raft_consensus.cc:2258] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:27.363502 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19330
I20260811 11:14:27.365027 19843 ts_tablet_manager.cc:1905] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:27.366650 19843 ts_tablet_manager.cc:1918] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:27.366786 19843 log.cc:1196] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/ae8daec9d82c4a5b8b1ef30dab5fd823
I20260811 11:14:27.367103 19843 ts_tablet_manager.cc:1939] T ae8daec9d82c4a5b8b1ef30dab5fd823 P e9951fa0ee8b4013be8438f133555dde: Deleting consensus metadata
I20260811 11:14:27.367383 19845 tablet_replica.cc:330] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: stopping tablet replica
I20260811 11:14:27.367506 19845 raft_consensus.cc:2229] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:27.367542 19230 catalog_manager.cc:4841] TS e9951fa0ee8b4013be8438f133555dde (127.29.145.3:39937): tablet ae8daec9d82c4a5b8b1ef30dab5fd823 (table test-table [id=54f2f49405e84bab958a44fec068ec5d]) successfully deleted
I20260811 11:14:27.367635 19845 raft_consensus.cc:2258] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:27.368314 19845 ts_tablet_manager.cc:1905] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:27.370272 19845 ts_tablet_manager.cc:1918] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:27.370353 19845 log.cc:1196] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/ae8daec9d82c4a5b8b1ef30dab5fd823
I20260811 11:14:27.370620 19845 ts_tablet_manager.cc:1939] T ae8daec9d82c4a5b8b1ef30dab5fd823 P 4b339f6a6aa941aeb33447de3c08c244: Deleting consensus metadata
I20260811 11:14:27.371125 19230 catalog_manager.cc:4841] TS 4b339f6a6aa941aeb33447de3c08c244 (127.29.145.2:46761): tablet ae8daec9d82c4a5b8b1ef30dab5fd823 (table test-table [id=54f2f49405e84bab958a44fec068ec5d]) successfully deleted
W20260811 11:14:27.373104 19231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:33395 (error 108)
I20260811 11:14:27.373687 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:27.373525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:33395 (local address 127.0.0.1:35362)
0811 11:14:27.373591 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:27.373649 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:33395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 11:14:27.373749 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19490
W20260811 11:14:27.382107 19231 catalog_manager.cc:4584] TS cddf6d0d0645468c966e72d3860dc18a (127.29.145.1:33395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae8daec9d82c4a5b8b1ef30dab5fd823: Network error: Client connection negotiation failed: client connection to 127.29.145.1:33395: connect: Connection refused (error 111)
I20260811 11:14:27.391999 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19649
I20260811 11:14:27.398691 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19212
2026-08-11T11:14:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (825 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:27 dist-test-slave-69rg krb5kdc[19849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:27 dist-test-slave-69rg krb5kdc[19849](info): set up 2 sockets
Aug 11 11:14:27 dist-test-slave-69rg krb5kdc[19849](info): commencing operation
krb5kdc: starting...
W20260811 11:14:28.529621 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:28 dist-test-slave-69rg krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:28.656773 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43251
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:43251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:28.744545 19864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:28.744740 19864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:28.744767 19864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:28.745923 19864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:28.745965 19864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:28.745978 19864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:28.745991 19864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:28.746003 19864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:28.747563 19864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:43251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:28.747828 19864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:28.748066 19864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:28.753110 19874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:28.753113 19869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:28.753139 19871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:28.753533 19864 server_base.cc:1034] running on GCE node
I20260811 11:14:28.753873 19864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:28.754184 19864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:28.755349 19864 hybrid_clock.cc:648] HybridClock initialized: now 1786446868755337 us; error 45 us; skew 500 ppm
Aug 11 11:14:28 dist-test-slave-69rg krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446868, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:28.762701 19864 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:28.763206 19864 webserver.cc:460] Webserver started at http://127.29.145.62:38007/ using document root <none> and password file <none>
I20260811 11:14:28.763468 19864 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:28.763535 19864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:28.763688 19864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:28.764765 19864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "08e231e17e2446a8bf4e228d8914b3fa"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "484d20d8b8e85a6d3e9d863a490fbb82"
server_key_iv: "6e5c6f1b4e4178038aba2360bdccc384"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.765125 19864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "08e231e17e2446a8bf4e228d8914b3fa"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "484d20d8b8e85a6d3e9d863a490fbb82"
server_key_iv: "6e5c6f1b4e4178038aba2360bdccc384"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.766568 19864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:28.767372 19881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:28.767575 19864 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:28.767679 19864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "08e231e17e2446a8bf4e228d8914b3fa"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "484d20d8b8e85a6d3e9d863a490fbb82"
server_key_iv: "6e5c6f1b4e4178038aba2360bdccc384"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.767764 19864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:28.776896 19864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:28.778834 19864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:28.779001 19864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:28.783228 19864 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:43251
I20260811 11:14:28.783262 19944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:43251 every 8 connection(s)
I20260811 11:14:28.783653 19864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:28.784540 19945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:28.786933 19945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: Bootstrap starting.
I20260811 11:14:28.787495 19945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:28.787762 19945 log.cc:824] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:28.788456 19945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: No bootstrap required, opened a new log
I20260811 11:14:28.790135 19945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } }
I20260811 11:14:28.790280 19945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:28.790304 19945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e231e17e2446a8bf4e228d8914b3fa, State: Initialized, Role: FOLLOWER
I20260811 11:14:28.790445 19945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } }
I20260811 11:14:28.790526 19945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:28.790550 19945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:28.790580 19945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:28.791085 19945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } }
I20260811 11:14:28.791172 19945 leader_election.cc:302] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08e231e17e2446a8bf4e228d8914b3fa; no voters:
I20260811 11:14:28.791306 19945 leader_election.cc:288] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:28.791396 19948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:28.791594 19948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [term 1 LEADER]: Becoming Leader. State: Replica: 08e231e17e2446a8bf4e228d8914b3fa, State: Running, Role: LEADER
I20260811 11:14:28.791615 19945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:28.791683 19948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } }
I20260811 11:14:28.792169 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19864
I20260811 11:14:28.792286 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:28.792263 19951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e231e17e2446a8bf4e228d8914b3fa. Latest consensus state: current_term: 1 leader_uuid: "08e231e17e2446a8bf4e228d8914b3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } } }
I20260811 11:14:28.792356 19951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:28.792508 30276 external_mini_cluster.cc:1428] Setting key 62670af292c2704714b7ac10632591a8
I20260811 11:14:28.792680 19950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08e231e17e2446a8bf4e228d8914b3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e231e17e2446a8bf4e228d8914b3fa" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43251 } } }
I20260811 11:14:28.792764 19950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:28.792829 19961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:28.793407 19961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:28.796159 19961 catalog_manager.cc:1347] Generated new cluster ID: 7e69f9b9586f44479e709737f1d30cc8
I20260811 11:14:28.796232 19961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:14:28 dist-test-slave-69rg krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:28.804275 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:28.793222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.29.145.62:43251)
0811 11:14:28.793675 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:28.793684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:28.793720 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:14:28.793875 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:28.793882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:28.793938 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:28.794079 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:28.794872 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.795757 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.796860 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.797474 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.797578 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:28.800443 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:28.800449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:28.800449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:28.800467 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:28.802629 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:28.803041 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:28.803045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:28.803045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:28.803165 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:28.803372 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:28.803373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:28.803374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:28.803690 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:28.803860 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:28.803930 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 11:14:28.804068 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":87,"threads_started":1}
I20260811 11:14:28.817014 19961 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:28.817535 19961 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:28.823660 19961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: Generated new TSK 0
I20260811 11:14:28.823849 19961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.29.145.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
Entry for principal kudu/127.29.145.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab.
I20260811 11:14:28.856458 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:28.939632 19977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:28.939832 19977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:28.939854 19977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:28.940971 19977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:28.941012 19977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:28.941037 19977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:28.942585 19977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.1.keytab
--principal=kudu/127.29.145.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:28.942831 19977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:28.943076 19977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:28.948239 19984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:28.948251 19982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:28.948256 19986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:28.948428 19977 server_base.cc:1034] running on GCE node
I20260811 11:14:28.948643 19977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:28.948892 19977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:28.950057 19977 hybrid_clock.cc:648] HybridClock initialized: now 1786446868950051 us; error 37 us; skew 500 ppm
Aug 11 11:14:28 dist-test-slave-69rg krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446868, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:28.952937 19977 init.cc:377] Logged in from keytab as kudu/127.29.145.1@KRBTEST.COM (short username kudu)
I20260811 11:14:28.953518 19977 webserver.cc:460] Webserver started at http://127.29.145.1:39369/ using document root <none> and password file <none>
I20260811 11:14:28.953770 19977 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:28.953876 19977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:28.954032 19977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:28.955080 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "23b7534140044597a4b7ccfc0c574c5f"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "710ca2ece62ce776c94475dda8a85963"
server_key_iv: "0ffa3bc04c35b88ef22329b89d471fe5"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.955490 19977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "23b7534140044597a4b7ccfc0c574c5f"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "710ca2ece62ce776c94475dda8a85963"
server_key_iv: "0ffa3bc04c35b88ef22329b89d471fe5"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.956928 19977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:28.957767 19993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:28.957981 19977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:28.958101 19977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "23b7534140044597a4b7ccfc0c574c5f"
format_stamp: "Formatted at 2026-08-11 11:14:28 on dist-test-slave-69rg"
server_key: "710ca2ece62ce776c94475dda8a85963"
server_key_iv: "0ffa3bc04c35b88ef22329b89d471fe5"
server_key_version: "encryptionkey@0"
I20260811 11:14:28.958211 19977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:28.975112 19977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:28.977222 19977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:28.977407 19977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:28.977658 19977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:28.978024 19977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:28.978102 19977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:28.978174 19977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:28.978206 19977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:28.984468 19977 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:43459
I20260811 11:14:28.984509 20122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:43459 every 8 connection(s)
I20260811 11:14:28.984861 19977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:14:28 dist-test-slave-69rg krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446868, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.1@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:28.992208 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 19977
I20260811 11:14:28.992359 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:28.992635 30276 external_mini_cluster.cc:1428] Setting key 5b2688c6cc06cd5ce36e5ff782827349
I20260811 11:14:28.995200 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:28.985661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43251 (local address 127.29.145.1:42939)
0811 11:14:28.985879 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:28.985916 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:28.986425 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:28.986688 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:28.986691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:28.987285 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:28.987479 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:28.987483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.988215 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.988216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:28.989010 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:28.989012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.989420 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.989421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:28.989546 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:28.990105 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:28.990114 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:28.991644 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:28.993804 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:28.993845 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:28.993847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:28.994086 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:28.994312 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:28.994313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:28.994313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:28.994413 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:28.994711 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:28.994716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:28.994842 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 11:14:28.994918 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:14:28.995038 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260811 11:14:28.995213 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:28.985697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:42939 (local address 127.29.145.62:43251)
0811 11:14:28.985771 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:28.985788 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:28.986012 (+ 224us) server_negotiation.cc:407] Connection header received
0811 11:14:28.986541 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:28.986542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:28.986556 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:28.986603 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:28.987584 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.988113 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.989107 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:28.989332 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:28.989410 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:28.991743 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:28.991745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:28.991745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:28.991752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:28.993720 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:28.994165 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:28.994167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:28.994167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:28.994236 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:28.994488 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:28.994489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:28.994489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:28.994650 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:28.994718 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:28.994982 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 11:14:28.995077 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:14:28.995919 20124 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43251
I20260811 11:14:28.996032 20124 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:28.996210 20124 heartbeater.cc:511] Master 127.29.145.62:43251 requested a full tablet report, sending...
I20260811 11:14:28.996760 19903 ts_manager.cc:194] Registered new tserver with Master: 23b7534140044597a4b7ccfc0c574c5f (127.29.145.1:43459)
I20260811 11:14:28.997411 19903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.1@KRBTEST.COM'} at 127.29.145.1:42939
WARNING: no policy specified for kudu/127.29.145.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
Entry for principal kudu/127.29.145.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab.
I20260811 11:14:29.021394 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:29.106663 20129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:29.106860 20129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:29.106884 20129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:29.107965 20129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:29.108000 20129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:29.108026 20129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:29.109426 20129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.2.keytab
--principal=kudu/127.29.145.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:29.109611 20129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:29.109782 20129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:29.114930 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.114946 20135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.115121 20134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.115116 20129 server_base.cc:1034] running on GCE node
I20260811 11:14:29.115427 20129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:29.115674 20129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:29.116816 20129 hybrid_clock.cc:648] HybridClock initialized: now 1786446869116796 us; error 45 us; skew 500 ppm
Aug 11 11:14:29 dist-test-slave-69rg krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446869, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:29.119472 20129 init.cc:377] Logged in from keytab as kudu/127.29.145.2@KRBTEST.COM (short username kudu)
I20260811 11:14:29.119956 20129 webserver.cc:460] Webserver started at http://127.29.145.2:40739/ using document root <none> and password file <none>
I20260811 11:14:29.120198 20129 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:29.120266 20129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:29.120424 20129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:29.121467 20129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "0a490be7a319487180b50c9bca2e40e5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "71138b4b53e452d876edcf9b07019e1c"
server_key_iv: "fe9f3e5797bd7f75883dde349cd7335b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.121902 20129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "0a490be7a319487180b50c9bca2e40e5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "71138b4b53e452d876edcf9b07019e1c"
server_key_iv: "fe9f3e5797bd7f75883dde349cd7335b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.123965 20129 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:14:29.125015 20146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.125223 20129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:29.125324 20129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "0a490be7a319487180b50c9bca2e40e5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "71138b4b53e452d876edcf9b07019e1c"
server_key_iv: "fe9f3e5797bd7f75883dde349cd7335b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.125399 20129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:29.137296 20129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:29.139662 20129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:29.139775 20129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:29.140003 20129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:29.140336 20129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:29.140393 20129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.140434 20129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:29.140458 20129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.146370 20129 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:36983
I20260811 11:14:29.146445 20284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:36983 every 8 connection(s)
I20260811 11:14:29.146768 20129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:29.147162 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20129
I20260811 11:14:29.147279 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:29.147516 30276 external_mini_cluster.cc:1428] Setting key 5b39a16179ce78f25cc7e5b12d2bb436
Aug 11 11:14:29 dist-test-slave-69rg krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446869, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.2@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.3@KRBTEST.COM; defaulting to no policy
I20260811 11:14:29.157503 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.147587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:47845 (local address 127.29.145.62:43251)
0811 11:14:29.147738 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.147740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.147911 (+ 171us) server_negotiation.cc:407] Connection header received
0811 11:14:29.148530 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.148532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.148546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.148617 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.149503 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.150327 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.151829 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.152144 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.152226 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.154559 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.154561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.154562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:29.154570 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.156029 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.156471 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.156472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.156473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.156536 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.156800 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.156801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.156802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.156925 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.157003 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.157321 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.157408 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 11:14:29.157701 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.147555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43251 (local address 127.29.145.2:47845)
0811 11:14:29.147772 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.147820 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.148430 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.148667 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.148670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.149228 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:29.149407 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.149411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.150428 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.150429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.151724 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.151727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.152260 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.152262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.152377 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.152934 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:29.152951 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:29.154456 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:29.156138 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:29.156145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:29.156147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:29.156388 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:29.156606 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:29.156607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:29.156607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:29.156720 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:29.156994 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:29.157001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:29.157179 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.157299 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.157469 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260811 11:14:29.158234 20285 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43251
I20260811 11:14:29.158324 20285 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:29.158463 20285 heartbeater.cc:511] Master 127.29.145.62:43251 requested a full tablet report, sending...
I20260811 11:14:29.158789 19903 ts_manager.cc:194] Registered new tserver with Master: 0a490be7a319487180b50c9bca2e40e5 (127.29.145.2:36983)
I20260811 11:14:29.159128 19903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.2@KRBTEST.COM'} at 127.29.145.2:47845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
Entry for principal kudu/127.29.145.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab.
I20260811 11:14:29.176921 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:29.260993 20290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:29.261281 20290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:29.261309 20290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:29.262498 20290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:29.262540 20290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:29.262566 20290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:29.264036 20290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.3.keytab
--principal=kudu/127.29.145.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:43251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:29.264294 20290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:29.264568 20290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:14:29.269979 20290 server_base.cc:1034] running on GCE node
W20260811 11:14:29.269982 20297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.270004 20298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.270005 20300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.270495 20290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:29.270763 20290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:29.271942 20290 hybrid_clock.cc:648] HybridClock initialized: now 1786446869271924 us; error 29 us; skew 500 ppm
Aug 11 11:14:29 dist-test-slave-69rg krb5kdc[19849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446869, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:29.274792 20290 init.cc:377] Logged in from keytab as kudu/127.29.145.3@KRBTEST.COM (short username kudu)
I20260811 11:14:29.275300 20290 webserver.cc:460] Webserver started at http://127.29.145.3:42191/ using document root <none> and password file <none>
I20260811 11:14:29.275540 20290 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:29.275631 20290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:29.275785 20290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:29.276880 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "dfe490fae8824e6ca841fcb1d1a67b9a"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5bfa45e32b3759fa99cb9e2156307d71"
server_key_iv: "53131634462eaea50c6651a3369f4c0c"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.277249 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "dfe490fae8824e6ca841fcb1d1a67b9a"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5bfa45e32b3759fa99cb9e2156307d71"
server_key_iv: "53131634462eaea50c6651a3369f4c0c"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.278712 20290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:29.279492 20309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.279685 20290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:29.279793 20290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "dfe490fae8824e6ca841fcb1d1a67b9a"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5bfa45e32b3759fa99cb9e2156307d71"
server_key_iv: "53131634462eaea50c6651a3369f4c0c"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.279881 20290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:29.293852 20290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:29.296192 20290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:29.296355 20290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:29.296598 20290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:29.296914 20290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:29.296968 20290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.297012 20290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:29.297037 20290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.303217 20290 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:39303
I20260811 11:14:29.303261 20438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:39303 every 8 connection(s)
I20260811 11:14:29.303580 20290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
Aug 11 11:14:29 dist-test-slave-69rg krb5kdc[19849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446869, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.3@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:29.312824 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20290
I20260811 11:14:29.312947 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:29.313148 30276 external_mini_cluster.cc:1428] Setting key 71d06fc9011d73d0b3e1b40b7c1a575b
I20260811 11:14:29.313952 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.304308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:48683 (local address 127.29.145.62:43251)
0811 11:14:29.304367 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.304369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.304620 (+ 251us) server_negotiation.cc:407] Connection header received
0811 11:14:29.305168 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.305170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.305181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.305240 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.306325 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.306868 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.308027 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.308233 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.308339 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.310543 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.310545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.310545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:29.310553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.312052 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.312487 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.312488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.312489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.312546 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.312990 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.312992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.312992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.313167 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.313277 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.313722 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.313849 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:14:29.314214 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.304221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43251 (local address 127.29.145.3:48683)
0811 11:14:29.304438 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.304495 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.305057 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.305294 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.305298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.305994 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:29.306218 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.306223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.306967 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.306968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.307796 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.307798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.308342 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.308343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.308461 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.308944 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:29.308953 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:29.310439 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:29.312174 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:29.312180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:29.312182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:29.312409 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:29.312683 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:29.312684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:29.312684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:29.312856 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:29.313320 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:29.313329 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:29.313526 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.313686 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.313908 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":96,"threads_started":1}
I20260811 11:14:29.314700 20439 heartbeater.cc:347] Connected to a master server at 127.29.145.62:43251
I20260811 11:14:29.314791 20439 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:29.314934 20439 heartbeater.cc:511] Master 127.29.145.62:43251 requested a full tablet report, sending...
I20260811 11:14:29.315275 19903 ts_manager.cc:194] Registered new tserver with Master: dfe490fae8824e6ca841fcb1d1a67b9a (127.29.145.3:39303)
I20260811 11:14:29.315696 19903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.29.145.3@KRBTEST.COM'} at 127.29.145.3:48683
I20260811 11:14:29.325531 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:29.334019 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.327093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.29.145.62:43251)
0811 11:14:29.327174 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.327175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.327241 (+ 66us) server_negotiation.cc:407] Connection header received
0811 11:14:29.327282 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.327283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.327292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.327343 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.328271 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.328836 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.329598 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.329791 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.329924 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.331081 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.331083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.331083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:29.331089 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.332554 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.332864 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.332866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.332866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.332917 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:29.333050 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:29.333050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:29.333051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:29.333229 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.333309 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.333714 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.333782 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:14:29.335955 19903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:29.337697 19903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:29.345463 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.340959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.29.145.2:36983)
0811 11:14:29.341141 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.341146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.341242 (+ 96us) server_negotiation.cc:407] Connection header received
0811 11:14:29.341352 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.341354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.341371 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.341446 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:29.341690 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.342573 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.344212 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.345044 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345187 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345253 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.345325 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.345369 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260811 11:14:29.345474 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.340654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36983 (local address 127.0.0.1:32830)
0811 11:14:29.341161 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.341171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.341253 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.341530 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.341532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.341534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:29.341607 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.341608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.342680 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.342681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.344105 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.344108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.345153 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.345221 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345226 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.345309 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.345386 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":76,"thread_start_us":65,"threads_started":1}
I20260811 11:14:29.345480 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.340650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:39303 (local address 127.0.0.1:53378)
0811 11:14:29.340769 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.340818 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.340911 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.341387 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.341391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.341393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:29.341471 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.341473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.342864 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.342865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.344150 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.344152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.345239 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.345301 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345305 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.345335 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.345376 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":15,"thread_start_us":103,"threads_started":1}
I20260811 11:14:29.345868 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.340933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.29.145.3:39303)
0811 11:14:29.341123 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.341129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.341141 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:14:29.341183 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.341185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.341202 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.341555 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:29.341634 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.342754 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.344439 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.345144 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345603 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345671 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.345698 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.345741 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260811 11:14:29.345906 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.341186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.29.145.1:43459)
0811 11:14:29.341407 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.341411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.341837 (+ 426us) server_negotiation.cc:407] Connection header received
0811 11:14:29.341894 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.341897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.341913 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.341997 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:29.342114 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.343059 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.344158 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.344933 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345008 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345071 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.345429 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.345472 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260811 11:14:29.346251 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.340650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:43459 (local address 127.0.0.1:43070)
0811 11:14:29.341731 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.341741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.341791 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.342012 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.342013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.342016 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:29.342054 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.342056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.343164 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.343165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.344065 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.344067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.345111 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.345112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.345206 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.345223 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.345299 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.345376 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260811 11:14:29.346688 20363 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb3230756be4a1383230587cc11b110 (DEFAULT_TABLE table=test-table [id=5fc343bfc412421d8d850fc88d0bd22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:29.347116 20363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb3230756be4a1383230587cc11b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:29.347002 20048 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb3230756be4a1383230587cc11b110 (DEFAULT_TABLE table=test-table [id=5fc343bfc412421d8d850fc88d0bd22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:29.347354 20048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb3230756be4a1383230587cc11b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:29.350303 20208 tablet_service.cc:1467] Processing CreateTablet for tablet 1cb3230756be4a1383230587cc11b110 (DEFAULT_TABLE table=test-table [id=5fc343bfc412421d8d850fc88d0bd22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:29.350678 20208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cb3230756be4a1383230587cc11b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:29.350832 20462 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: Bootstrap starting.
I20260811 11:14:29.351480 20462 tablet_bootstrap.cc:654] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:29.352046 20462 log.cc:824] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:29.353071 20462 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: No bootstrap required, opened a new log
I20260811 11:14:29.353257 20462 ts_tablet_manager.cc:1397] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:29.353760 20464 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Bootstrap starting.
I20260811 11:14:29.354578 20464 tablet_bootstrap.cc:654] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:29.354590 20466 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Bootstrap starting.
I20260811 11:14:29.355181 20466 tablet_bootstrap.cc:654] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:29.355257 20464 log.cc:824] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:29.355527 20462 raft_consensus.cc:348] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.355722 20462 raft_consensus.cc:374] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:29.355767 20462 raft_consensus.cc:729] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b7534140044597a4b7ccfc0c574c5f, State: Initialized, Role: FOLLOWER
I20260811 11:14:29.355818 20466 log.cc:824] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:29.355904 20462 consensus_queue.cc:260] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.356208 20464 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: No bootstrap required, opened a new log
I20260811 11:14:29.356230 20462 ts_tablet_manager.cc:1428] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 11:14:29.356282 20464 ts_tablet_manager.cc:1397] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:14:29.356354 20124 heartbeater.cc:503] Master 127.29.145.62:43251 was elected leader, sending a full tablet report...
I20260811 11:14:29.356634 20466 tablet_bootstrap.cc:492] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: No bootstrap required, opened a new log
I20260811 11:14:29.356714 20466 ts_tablet_manager.cc:1397] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:29.357791 20464 raft_consensus.cc:348] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.358017 20464 raft_consensus.cc:374] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:29.358065 20464 raft_consensus.cc:729] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a490be7a319487180b50c9bca2e40e5, State: Initialized, Role: FOLLOWER
I20260811 11:14:29.358203 20464 consensus_queue.cc:260] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.358311 20466 raft_consensus.cc:348] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.358479 20466 raft_consensus.cc:374] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:29.358516 20466 raft_consensus.cc:729] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe490fae8824e6ca841fcb1d1a67b9a, State: Initialized, Role: FOLLOWER
I20260811 11:14:29.358515 20464 ts_tablet_manager.cc:1428] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:29.358532 20285 heartbeater.cc:503] Master 127.29.145.62:43251 was elected leader, sending a full tablet report...
I20260811 11:14:29.358613 20466 consensus_queue.cc:260] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.358880 20466 ts_tablet_manager.cc:1428] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:14:29.359032 20439 heartbeater.cc:503] Master 127.29.145.62:43251 was elected leader, sending a full tablet report...
W20260811 11:14:29.397552 20286 tablet.cc:2369] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:29.437565 20469 raft_consensus.cc:482] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:29.437752 20469 raft_consensus.cc:504] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.438354 20469 leader_election.cc:288] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfe490fae8824e6ca841fcb1d1a67b9a (127.29.145.3:39303), 0a490be7a319487180b50c9bca2e40e5 (127.29.145.2:36983)
I20260811 11:14:29.441288 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.438523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:39303 (local address 127.29.145.1:48907)
0811 11:14:29.438583 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.438595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.438642 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.438944 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.438946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.438948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:29.439025 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.439027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.439682 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.439683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.440429 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.440431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.441008 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.441009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.441094 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.441097 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.441176 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.441217 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:14:29.441288 20460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.438701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:48907 (local address 127.29.145.3:39303)
0811 11:14:29.438792 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.438795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.438805 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:29.438835 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.438836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.438853 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.438911 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:29.439110 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.439587 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.440536 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.440922 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.441009 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.441018 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.441187 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.441226 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 11:14:29.441530 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.438523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:36983 (local address 127.29.145.1:50467)
0811 11:14:29.438983 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.438994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.439073 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.439217 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.439218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.439219 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:29.439283 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.439285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.439928 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.439929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.440656 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.440657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.441227 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.441228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.441299 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.441310 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.441382 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.441432 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":20,"thread_start_us":383,"threads_started":1}
I20260811 11:14:29.441622 20459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.438713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:50467 (local address 127.29.145.2:36983)
0811 11:14:29.438781 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.438783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.439068 (+ 285us) server_negotiation.cc:407] Connection header received
0811 11:14:29.439152 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.439153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.439162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.439206 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:29.439360 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.439842 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.440748 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.441130 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.441223 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.441231 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.441503 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.441543 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:14:29.441917 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb3230756be4a1383230587cc11b110" candidate_uuid: "23b7534140044597a4b7ccfc0c574c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" is_pre_election: true
I20260811 11:14:29.441975 20234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb3230756be4a1383230587cc11b110" candidate_uuid: "23b7534140044597a4b7ccfc0c574c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a490be7a319487180b50c9bca2e40e5" is_pre_election: true
I20260811 11:14:29.442087 20387 raft_consensus.cc:2454] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b7534140044597a4b7ccfc0c574c5f in term 0.
I20260811 11:14:29.442129 20234 raft_consensus.cc:2454] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b7534140044597a4b7ccfc0c574c5f in term 0.
I20260811 11:14:29.442363 19994 leader_election.cc:302] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a490be7a319487180b50c9bca2e40e5, 23b7534140044597a4b7ccfc0c574c5f; no voters:
I20260811 11:14:29.442521 20469 raft_consensus.cc:2781] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:29.442582 20469 raft_consensus.cc:482] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:29.442600 20469 raft_consensus.cc:3037] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:29.443282 20469 raft_consensus.cc:504] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.443423 20469 leader_election.cc:288] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [CANDIDATE]: Term 1 election: Requested vote from peers dfe490fae8824e6ca841fcb1d1a67b9a (127.29.145.3:39303), 0a490be7a319487180b50c9bca2e40e5 (127.29.145.2:36983)
I20260811 11:14:29.443575 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb3230756be4a1383230587cc11b110" candidate_uuid: "23b7534140044597a4b7ccfc0c574c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a"
I20260811 11:14:29.443694 20387 raft_consensus.cc:3037] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:29.443783 20234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cb3230756be4a1383230587cc11b110" candidate_uuid: "23b7534140044597a4b7ccfc0c574c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a490be7a319487180b50c9bca2e40e5"
I20260811 11:14:29.443909 20234 raft_consensus.cc:3037] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:29.444497 20387 raft_consensus.cc:2454] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b7534140044597a4b7ccfc0c574c5f in term 1.
I20260811 11:14:29.444767 19997 leader_election.cc:302] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b7534140044597a4b7ccfc0c574c5f, dfe490fae8824e6ca841fcb1d1a67b9a; no voters:
I20260811 11:14:29.444783 20234 raft_consensus.cc:2454] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b7534140044597a4b7ccfc0c574c5f in term 1.
I20260811 11:14:29.444866 20469 raft_consensus.cc:2781] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:29.445030 20469 raft_consensus.cc:686] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 1 LEADER]: Becoming Leader. State: Replica: 23b7534140044597a4b7ccfc0c574c5f, State: Running, Role: LEADER
I20260811 11:14:29.445163 20469 consensus_queue.cc:237] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } }
I20260811 11:14:29.446079 19903 catalog_manager.cc:5495] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b7534140044597a4b7ccfc0c574c5f (127.29.145.1). New cstate: current_term: 1 leader_uuid: "23b7534140044597a4b7ccfc0c574c5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b7534140044597a4b7ccfc0c574c5f" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 43459 } health_report { overall_health: HEALTHY } } }
I20260811 11:14:29.463513 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.460884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.29.145.1:43459)
0811 11:14:29.460931 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.460934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.460944 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:29.461036 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.461037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.461056 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.461121 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:29.461286 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.461897 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.462665 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.462878 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.462956 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.463154 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:29.463263 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:29.463352 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.463392 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.463414 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:14:29.465456 20387 raft_consensus.cc:1261] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 1 FOLLOWER]: Refusing update from remote peer 23b7534140044597a4b7ccfc0c574c5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:29.465476 20234 raft_consensus.cc:1261] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 1 FOLLOWER]: Refusing update from remote peer 23b7534140044597a4b7ccfc0c574c5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:29.465883 20473 consensus_queue.cc:1035] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfe490fae8824e6ca841fcb1d1a67b9a" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 39303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:29.466033 20469 consensus_queue.cc:1035] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a490be7a319487180b50c9bca2e40e5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:29.469142 20479 mvcc.cc:205] Tried to move back new op lower bound from 7317286377327874048 to 7317286377248088064. Current Snapshot: MvccSnapshot[applied={T|T < 7317286377327874048}]
I20260811 11:14:29.468775 20477 mvcc.cc:205] Tried to move back new op lower bound from 7317286377327874048 to 7317286377248088064. Current Snapshot: MvccSnapshot[applied={T|T < 7317286377327874048}]
I20260811 11:14:29.472087 19903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:29.472170 19903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:29.472980 19903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08e231e17e2446a8bf4e228d8914b3fa: Sending DeleteTablet for 3 replicas of tablet 1cb3230756be4a1383230587cc11b110
I20260811 11:14:29.473335 20048 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb3230756be4a1383230587cc11b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:29 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:43070
I20260811 11:14:29.473352 20363 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb3230756be4a1383230587cc11b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:29 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:53378
I20260811 11:14:29.473362 20208 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cb3230756be4a1383230587cc11b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:29 UTC) from {username='kudu', principal='kudu/127.29.145.62@KRBTEST.COM'} at 127.0.0.1:32830
I20260811 11:14:29.473639 20489 tablet_replica.cc:330] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f: stopping tablet replica
I20260811 11:14:29.473737 20490 tablet_replica.cc:330] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: stopping tablet replica
I20260811 11:14:29.473747 20489 raft_consensus.cc:2229] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:14:29.473889 20489 raft_consensus.cc:2258] T 1cb3230756be4a1383230587cc11b110 P 23b7534140044597a4b7ccfc0c574c5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:29.473917 20490 raft_consensus.cc:2229] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:29.473981 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19977
I20260811 11:14:29.474570 20488 tablet_replica.cc:330] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: stopping tablet replica
I20260811 11:14:29.474603 20490 raft_consensus.cc:2258] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:29.474864 20488 raft_consensus.cc:2229] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:29.475020 20488 raft_consensus.cc:2258] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:29.475919 20488 ts_tablet_manager.cc:1905] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:29.476718 20490 ts_tablet_manager.cc:1905] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:29.477936 20488 ts_tablet_manager.cc:1918] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:29.478009 20488 log.cc:1196] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/1cb3230756be4a1383230587cc11b110
I20260811 11:14:29.478271 20488 ts_tablet_manager.cc:1939] T 1cb3230756be4a1383230587cc11b110 P dfe490fae8824e6ca841fcb1d1a67b9a: Deleting consensus metadata
I20260811 11:14:29.478470 20490 ts_tablet_manager.cc:1918] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:29.478641 20490 log.cc:1196] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/1cb3230756be4a1383230587cc11b110
I20260811 11:14:29.478971 20490 ts_tablet_manager.cc:1939] T 1cb3230756be4a1383230587cc11b110 P 0a490be7a319487180b50c9bca2e40e5: Deleting consensus metadata
I20260811 11:14:29.478955 19885 catalog_manager.cc:4841] TS dfe490fae8824e6ca841fcb1d1a67b9a (127.29.145.3:39303): tablet 1cb3230756be4a1383230587cc11b110 (table test-table [id=5fc343bfc412421d8d850fc88d0bd22e]) successfully deleted
I20260811 11:14:29.479444 19882 catalog_manager.cc:4841] TS 0a490be7a319487180b50c9bca2e40e5 (127.29.145.2:36983): tablet 1cb3230756be4a1383230587cc11b110 (table test-table [id=5fc343bfc412421d8d850fc88d0bd22e]) successfully deleted
W20260811 11:14:29.482849 19884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:43459 (error 108)
I20260811 11:14:29.483378 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20129
I20260811 11:14:29.483407 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.483250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:43459 (local address 127.0.0.1:43078)
0811 11:14:29.483327 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.483369 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:43459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 11:14:29.483530 19884 catalog_manager.cc:4584] TS 23b7534140044597a4b7ccfc0c574c5f (127.29.145.1:43459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1cb3230756be4a1383230587cc11b110: Network error: Client connection negotiation failed: client connection to 127.29.145.1:43459: connect: Connection refused (error 111)
I20260811 11:14:29.490466 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20290
I20260811 11:14:29.497766 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 19864
2026-08-11T11:14:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2094 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T11:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:29Z Disabled control of system clock
I20260811 11:14:29.530370 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42067
--webserver_interface=127.29.145.62
--webserver_port=0
--builtin_ntp_servers=127.29.145.20:33325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:29.614024 20496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:29.614259 20496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:29.614280 20496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:29.615476 20496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:29.615520 20496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:29.615533 20496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:29.615545 20496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:29.615557 20496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:29.616868 20496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:29.617069 20496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:29.617352 20496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:29.622702 20503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.622797 20506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.622798 20504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.622939 20496 server_base.cc:1034] running on GCE node
I20260811 11:14:29.623229 20496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:29.623546 20496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:29.624725 20496 hybrid_clock.cc:648] HybridClock initialized: now 1786446869624700 us; error 42 us; skew 500 ppm
I20260811 11:14:29.626631 20496 webserver.cc:460] Webserver started at http://127.29.145.62:42987/ using document root <none> and password file <none>
I20260811 11:14:29.626931 20496 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:29.627012 20496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:29.627174 20496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:29.628291 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "8f4656a1270d442681abee365cc969ed"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "2db2ab251a04f9233f3300a6145846d3"
server_key_iv: "77dd7cc21589c46b68df48578d8b627b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.628674 20496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "8f4656a1270d442681abee365cc969ed"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "2db2ab251a04f9233f3300a6145846d3"
server_key_iv: "77dd7cc21589c46b68df48578d8b627b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.630219 20496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260811 11:14:29.630993 20512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.631201 20496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:29.631316 20496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "8f4656a1270d442681abee365cc969ed"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "2db2ab251a04f9233f3300a6145846d3"
server_key_iv: "77dd7cc21589c46b68df48578d8b627b"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.631393 20496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:29.648625 20496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:29.648960 20496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:29.649133 20496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:29.653316 20496 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42067
I20260811 11:14:29.653355 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42067 every 8 connection(s)
I20260811 11:14:29.653743 20496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:29.655045 20572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:29.655350 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20496
I20260811 11:14:29.655465 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:29.655689 30276 external_mini_cluster.cc:1428] Setting key 0798810f302ed30915192a8c3e726cf9
I20260811 11:14:29.658665 20572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: Bootstrap starting.
I20260811 11:14:29.659370 20572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:29.659672 20572 log.cc:824] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:29.660367 20572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: No bootstrap required, opened a new log
I20260811 11:14:29.661537 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.656271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.29.145.62:42067)
0811 11:14:29.656476 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.656484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.656509 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:14:29.656968 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.656974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.657033 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.657162 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.657417 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.658367 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.659467 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.659926 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.660069 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.660867 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.660871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.660871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:29.661041 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.661072 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:29.661082 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.661210 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.661281 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.661363 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260811 11:14:29.662426 20572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } }
I20260811 11:14:29.662586 20572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:29.662612 20572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f4656a1270d442681abee365cc969ed, State: Initialized, Role: FOLLOWER
I20260811 11:14:29.662732 20572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } }
I20260811 11:14:29.662807 20572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:29.662850 20572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:29.662900 20572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:29.663465 20572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } }
I20260811 11:14:29.663578 20572 leader_election.cc:302] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f4656a1270d442681abee365cc969ed; no voters:
I20260811 11:14:29.663774 20572 leader_election.cc:288] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:29.663852 20580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:29.664002 20580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [term 1 LEADER]: Becoming Leader. State: Replica: 8f4656a1270d442681abee365cc969ed, State: Running, Role: LEADER
I20260811 11:14:29.664045 20572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:29.664119 20580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } }
I20260811 11:14:29.664492 20582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f4656a1270d442681abee365cc969ed. Latest consensus state: current_term: 1 leader_uuid: "8f4656a1270d442681abee365cc969ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } } }
I20260811 11:14:29.664589 20582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:29.664487 20581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f4656a1270d442681abee365cc969ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f4656a1270d442681abee365cc969ed" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42067 } } }
I20260811 11:14:29.664736 20581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:29.664855 20589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:29.665323 20589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:29.666792 20589 catalog_manager.cc:1347] Generated new cluster ID: 84bfbf01a57740a5bbcfbc9cf224330c
I20260811 11:14:29.666849 20589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:29.687570 20589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:29.688107 20589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:29.692883 20589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: Generated new TSK 0
I20260811 11:14:29.693080 20589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:29.700073 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.1:0
--local_ip_for_outbound_sockets=127.29.145.1
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--builtin_ntp_servers=127.29.145.20:33325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:29.785534 20605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:29.785774 20605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:29.785810 20605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:29.786970 20605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:29.787032 20605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:29.787075 20605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.1
I20260811 11:14:29.788427 20605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.29.145.1
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.1
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:29.788694 20605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:29.788995 20605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:29.795094 20611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.795115 20613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.795164 20605 server_base.cc:1034] running on GCE node
W20260811 11:14:29.795082 20616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.795540 20605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:29.795758 20605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:29.796903 20605 hybrid_clock.cc:648] HybridClock initialized: now 1786446869796891 us; error 30 us; skew 500 ppm
I20260811 11:14:29.798177 20605 webserver.cc:460] Webserver started at http://127.29.145.1:34449/ using document root <none> and password file <none>
I20260811 11:14:29.798395 20605 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:29.798436 20605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:29.798530 20605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:29.799913 20605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data/instance:
uuid: "abb98461d728404dad141ae35c5d7ec1"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "4684f8cc806b522fe7d80cf000ba0a12"
server_key_iv: "ac77724176542eff4ff5ad3d61374da9"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.800240 20605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance:
uuid: "abb98461d728404dad141ae35c5d7ec1"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "4684f8cc806b522fe7d80cf000ba0a12"
server_key_iv: "ac77724176542eff4ff5ad3d61374da9"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.801579 20605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:29.802419 20622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.802598 20605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:29.802685 20605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal
uuid: "abb98461d728404dad141ae35c5d7ec1"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "4684f8cc806b522fe7d80cf000ba0a12"
server_key_iv: "ac77724176542eff4ff5ad3d61374da9"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.802742 20605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:29.821010 20605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:29.821309 20605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:29.821416 20605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:29.821599 20605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:29.821907 20605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:29.821943 20605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.821965 20605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:29.821977 20605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.827723 20605 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.1:36087
I20260811 11:14:29.827804 20753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.1:36087 every 8 connection(s)
I20260811 11:14:29.828130 20605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/data/info.pb
I20260811 11:14:29.833292 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.828970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.1:55751 (local address 127.29.145.62:42067)
0811 11:14:29.829028 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.829031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.829261 (+ 230us) server_negotiation.cc:407] Connection header received
0811 11:14:29.829797 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.829798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.829852 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.829919 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.830565 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.831066 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.832021 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.832203 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.832256 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.833051 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.833052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.833052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:29.833057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.833071 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:29.833075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.833118 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.833201 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.833228 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 11:14:29.833531 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.828924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42067 (local address 127.29.145.1:55751)
0811 11:14:29.829127 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.829166 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.829699 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.829983 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.829986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.829994 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:29.830493 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.830496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.831152 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.831152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.831945 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.831947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.832282 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.832283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.832387 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.832918 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:29.832918 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:29.832947 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:29.832968 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:29.833131 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:29.833136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:29.833138 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.833195 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.833287 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260811 11:14:29.834185 20754 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42067
I20260811 11:14:29.834298 20754 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:29.834482 20754 heartbeater.cc:511] Master 127.29.145.62:42067 requested a full tablet report, sending...
I20260811 11:14:29.835057 20531 ts_manager.cc:194] Registered new tserver with Master: abb98461d728404dad141ae35c5d7ec1 (127.29.145.1:36087)
I20260811 11:14:29.835736 20531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.1:55751
I20260811 11:14:29.836061 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20605
I20260811 11:14:29.836176 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-0/wal/instance
I20260811 11:14:29.836390 30276 external_mini_cluster.cc:1428] Setting key 6caed2e6aa417805cdf226da2a902038
I20260811 11:14:29.837271 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.2:0
--local_ip_for_outbound_sockets=127.29.145.2
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--builtin_ntp_servers=127.29.145.20:33325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:14:29.923013 20757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:29.923233 20757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:29.923255 20757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:29.924319 20757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:29.924351 20757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:29.924376 20757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.2
I20260811 11:14:29.925634 20757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.29.145.2
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.2
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:29.925869 20757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:29.926143 20757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:29.931329 20762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.931337 20763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:29.931329 20766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:29.931769 20757 server_base.cc:1034] running on GCE node
I20260811 11:14:29.931991 20757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:29.932271 20757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:29.933452 20757 hybrid_clock.cc:648] HybridClock initialized: now 1786446869933453 us; error 44 us; skew 500 ppm
I20260811 11:14:29.934857 20757 webserver.cc:460] Webserver started at http://127.29.145.2:44813/ using document root <none> and password file <none>
I20260811 11:14:29.935101 20757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:29.935180 20757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:29.935340 20757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:29.936335 20757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data/instance:
uuid: "4c9ac302511e48239bf6fb88f3b3d1a5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5d3de40b9a081c76df8326317a354307"
server_key_iv: "cf20c75a6aa1d26632851ea0010bd6b3"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.936635 20757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance:
uuid: "4c9ac302511e48239bf6fb88f3b3d1a5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5d3de40b9a081c76df8326317a354307"
server_key_iv: "cf20c75a6aa1d26632851ea0010bd6b3"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.937932 20757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:29.938684 20773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.938874 20757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:14:29.938958 20757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal
uuid: "4c9ac302511e48239bf6fb88f3b3d1a5"
format_stamp: "Formatted at 2026-08-11 11:14:29 on dist-test-slave-69rg"
server_key: "5d3de40b9a081c76df8326317a354307"
server_key_iv: "cf20c75a6aa1d26632851ea0010bd6b3"
server_key_version: "encryptionkey@0"
I20260811 11:14:29.939049 20757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:29.952256 20757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:29.952615 20757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:29.952740 20757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:29.952986 20757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:29.953253 20757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:29.953284 20757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.953305 20757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:29.953317 20757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:29.959473 20757 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.2:33133
I20260811 11:14:29.959579 20900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.2:33133 every 8 connection(s)
I20260811 11:14:29.959895 20757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/data/info.pb
I20260811 11:14:29.963095 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20757
I20260811 11:14:29.963187 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal/instance
I20260811 11:14:29.963419 30276 external_mini_cluster.cc:1428] Setting key 7717ce21b022365cf5a90c1b501f692d
I20260811 11:14:29.964430 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.29.145.3:0
--local_ip_for_outbound_sockets=127.29.145.3
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--builtin_ntp_servers=127.29.145.20:33325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:14:29.965399 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.960715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:39583 (local address 127.29.145.62:42067)
0811 11:14:29.960761 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:29.960763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:29.960981 (+ 218us) server_negotiation.cc:407] Connection header received
0811 11:14:29.961445 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:29.961446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:29.961456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:29.961506 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:29.962255 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.962756 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.963871 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.964081 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.964156 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.965144 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:29.965145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:29.965146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:29.965150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:29.965164 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:29.965169 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:29.965217 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:29.965294 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 11:14:29.965321 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:14:29.965615 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:29.960670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42067 (local address 127.29.145.2:39583)
0811 11:14:29.960864 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:29.960896 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:29.961363 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:29.961556 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:29.961559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:29.961568 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:29.962145 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.962150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.962851 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.962852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.963769 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:29.963772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:29.964165 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:29.964166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:29.964348 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:29.964974 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:29.964975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:29.965018 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:29.965039 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:29.965216 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:29.965221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:29.965224 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:14:29.965296 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:14:29.965393 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260811 11:14:29.966220 20901 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42067
I20260811 11:14:29.966310 20901 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:29.966500 20901 heartbeater.cc:511] Master 127.29.145.62:42067 requested a full tablet report, sending...
I20260811 11:14:29.966912 20531 ts_manager.cc:194] Registered new tserver with Master: 4c9ac302511e48239bf6fb88f3b3d1a5 (127.29.145.2:33133)
I20260811 11:14:29.967316 20531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.2:39583
W20260811 11:14:30.050057 20904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:30.050303 20904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:30.050341 20904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:30.051496 20904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:30.051564 20904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:30.051609 20904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.29.145.3
I20260811 11:14:30.052994 20904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.29.145.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.29.145.3
--webserver_port=0
--tserver_master_addrs=127.29.145.62:42067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.29.145.3
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:30.053274 20904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:30.053594 20904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:30.059419 20910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:30.059504 20914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:30.059551 20904 server_base.cc:1034] running on GCE node
W20260811 11:14:30.059649 20911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:30.059973 20904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:30.060300 20904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:30.061427 20904 hybrid_clock.cc:648] HybridClock initialized: now 1786446870061394 us; error 48 us; skew 500 ppm
I20260811 11:14:30.063203 20904 webserver.cc:460] Webserver started at http://127.29.145.3:37013/ using document root <none> and password file <none>
I20260811 11:14:30.063511 20904 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:30.063566 20904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:30.063712 20904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:30.065119 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data/instance:
uuid: "41dc8722d78e460cb1c13c8e2e03f89e"
format_stamp: "Formatted at 2026-08-11 11:14:30 on dist-test-slave-69rg"
server_key: "01fb2786b595fd56730b44da20e40edc"
server_key_iv: "a383a554d6e33e89f3086685cb7e7532"
server_key_version: "encryptionkey@0"
I20260811 11:14:30.065575 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance:
uuid: "41dc8722d78e460cb1c13c8e2e03f89e"
format_stamp: "Formatted at 2026-08-11 11:14:30 on dist-test-slave-69rg"
server_key: "01fb2786b595fd56730b44da20e40edc"
server_key_iv: "a383a554d6e33e89f3086685cb7e7532"
server_key_version: "encryptionkey@0"
I20260811 11:14:30.067281 20904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:30.068455 20921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:30.068642 20904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:30.068758 20904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal
uuid: "41dc8722d78e460cb1c13c8e2e03f89e"
format_stamp: "Formatted at 2026-08-11 11:14:30 on dist-test-slave-69rg"
server_key: "01fb2786b595fd56730b44da20e40edc"
server_key_iv: "a383a554d6e33e89f3086685cb7e7532"
server_key_version: "encryptionkey@0"
I20260811 11:14:30.068850 20904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:30.083917 20904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:30.084227 20904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:30.084373 20904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:30.084607 20904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:14:30.084983 20904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:14:30.085042 20904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:30.085086 20904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:14:30.085114 20904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:30.091020 20904 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.3:33721
I20260811 11:14:30.091115 21056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.3:33721 every 8 connection(s)
I20260811 11:14:30.091445 20904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/data/info.pb
I20260811 11:14:30.096441 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.092234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.3:37303 (local address 127.29.145.62:42067)
0811 11:14:30.092305 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.092307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.092512 (+ 205us) server_negotiation.cc:407] Connection header received
0811 11:14:30.092970 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.092971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.092982 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.093044 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:30.093712 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.094230 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.095194 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.095382 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.095440 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.096210 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:30.096211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:30.096211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:30.096215 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:30.096231 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:30.096235 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:30.096281 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.096359 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.096381 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:14:30.096654 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.092189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:42067 (local address 127.29.145.3:37303)
0811 11:14:30.092380 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.092420 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.092887 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.093112 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.093116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.093124 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:30.093617 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.093621 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.094319 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.094319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.095091 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.095093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.095464 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.095465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.095572 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.096077 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:14:30.096077 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:30.096104 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:14:30.096126 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:30.096288 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:30.096291 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:30.096293 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.096342 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.096419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260811 11:14:30.097126 21057 heartbeater.cc:347] Connected to a master server at 127.29.145.62:42067
I20260811 11:14:30.097261 21057 heartbeater.cc:464] Registering TS with master...
I20260811 11:14:30.097466 21057 heartbeater.cc:511] Master 127.29.145.62:42067 requested a full tablet report, sending...
I20260811 11:14:30.097862 20531 ts_manager.cc:194] Registered new tserver with Master: 41dc8722d78e460cb1c13c8e2e03f89e (127.29.145.3:33721)
I20260811 11:14:30.098264 20531 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.29.145.3:37303
I20260811 11:14:30.101389 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 20904
I20260811 11:14:30.101521 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal/instance
I20260811 11:14:30.101763 30276 external_mini_cluster.cc:1428] Setting key 2bd10dac9fbfd77c59216ef00ace24f6
I20260811 11:14:30.102792 30276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:14:30.107299 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.104227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.29.145.62:42067)
0811 11:14:30.104297 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.104299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.104396 (+ 97us) server_negotiation.cc:407] Connection header received
0811 11:14:30.104434 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.104434 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.104443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.104485 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:30.104672 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.105198 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.106069 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.106260 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.106319 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.107057 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:30.107058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:30.107058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:30.107062 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:30.107075 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:30.107079 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:30.107137 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.107174 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.107193 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:14:30.109206 20531 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:14:30.110904 20531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:14:30.117491 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:33721 (local address 127.0.0.1:46442)
0811 11:14:30.113950 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.113989 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.114066 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.114349 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.114352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.114354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:30.114432 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.114433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115405 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.115406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.116465 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.116468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.117158 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.117159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.117242 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.117257 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.117361 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.117416 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260811 11:14:30.117502 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.29.145.3:33721)
0811 11:14:30.113865 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.113869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.114048 (+ 179us) server_negotiation.cc:407] Connection header received
0811 11:14:30.114173 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.114176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.114192 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.114273 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:30.114518 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115318 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.116548 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.117055 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.117149 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.117240 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.117379 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.117418 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":95,"threads_started":1}
I20260811 11:14:30.118527 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:36087 (local address 127.0.0.1:37468)
0811 11:14:30.114323 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.114332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.114369 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.114652 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.114653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.114655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:30.114687 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.114688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115973 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.115974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.116767 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.116769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.118257 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.118258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.118327 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.118331 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.118402 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.118447 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":67,"threads_started":1}
I20260811 11:14:30.118567 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.29.145.1:36087)
0811 11:14:30.114493 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.114498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.114507 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:30.114544 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.114546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.114563 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.114643 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:30.114762 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115885 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.117403 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.118165 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.118245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.118338 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.118418 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.118458 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":137,"threads_started":1}
I20260811 11:14:30.118461 20978 tablet_service.cc:1467] Processing CreateTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 (DEFAULT_TABLE table=test-table [id=fa2a70dfecbd481981e56b33b83bbcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:30.118826 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.2:33133 (local address 127.0.0.1:58536)
0811 11:14:30.114188 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.114197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.114245 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.114423 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.114424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.114426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:30.114459 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.114460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115743 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.115744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.117269 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.117270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.118408 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.118409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.118475 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.118479 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.118532 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.118741 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":781,"thread_start_us":90,"threads_started":1}
I20260811 11:14:30.118916 20978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce12312b52be4bd997d29fa81c4c1d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:30.119058 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.113498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58536 (local address 127.29.145.2:33133)
0811 11:14:30.113756 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.113763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.114258 (+ 495us) server_negotiation.cc:407] Connection header received
0811 11:14:30.114310 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.114312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.114330 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.114413 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:30.114525 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.115647 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.117363 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.118234 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.118296 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.118369 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.118937 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.118981 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":116,"threads_started":1}
I20260811 11:14:30.119227 20680 tablet_service.cc:1467] Processing CreateTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 (DEFAULT_TABLE table=test-table [id=fa2a70dfecbd481981e56b33b83bbcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:30.119539 20680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce12312b52be4bd997d29fa81c4c1d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:30.121685 21075 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Bootstrap starting.
I20260811 11:14:30.122462 21076 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Bootstrap starting.
I20260811 11:14:30.122532 21075 tablet_bootstrap.cc:654] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:30.122565 20827 tablet_service.cc:1467] Processing CreateTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 (DEFAULT_TABLE table=test-table [id=fa2a70dfecbd481981e56b33b83bbcee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:14:30.122890 21075 log.cc:824] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:30.122938 20827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce12312b52be4bd997d29fa81c4c1d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:30.123248 21076 tablet_bootstrap.cc:654] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:30.123571 21076 log.cc:824] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:30.123621 21075 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: No bootstrap required, opened a new log
I20260811 11:14:30.123705 21075 ts_tablet_manager.cc:1397] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:14:30.125895 21080 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Bootstrap starting.
I20260811 11:14:30.125789 21075 raft_consensus.cc:348] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.125977 21075 raft_consensus.cc:374] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:30.126019 21075 raft_consensus.cc:729] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41dc8722d78e460cb1c13c8e2e03f89e, State: Initialized, Role: FOLLOWER
I20260811 11:14:30.126165 21075 consensus_queue.cc:260] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.126439 21075 ts_tablet_manager.cc:1428] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 11:14:30.126645 21076 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: No bootstrap required, opened a new log
I20260811 11:14:30.126718 21076 ts_tablet_manager.cc:1397] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 11:14:30.127615 21057 heartbeater.cc:503] Master 127.29.145.62:42067 was elected leader, sending a full tablet report...
I20260811 11:14:30.130111 21076 raft_consensus.cc:348] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.130287 21076 raft_consensus.cc:374] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:30.130311 21076 raft_consensus.cc:729] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb98461d728404dad141ae35c5d7ec1, State: Initialized, Role: FOLLOWER
I20260811 11:14:30.130451 21076 consensus_queue.cc:260] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.130748 21076 ts_tablet_manager.cc:1428] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 11:14:30.130918 20754 heartbeater.cc:503] Master 127.29.145.62:42067 was elected leader, sending a full tablet report...
I20260811 11:14:30.131294 21080 tablet_bootstrap.cc:654] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:30.131656 21080 log.cc:824] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:30.132522 21080 tablet_bootstrap.cc:492] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: No bootstrap required, opened a new log
I20260811 11:14:30.132584 21080 ts_tablet_manager.cc:1397] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 11:14:30.134732 21080 raft_consensus.cc:348] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.134861 21080 raft_consensus.cc:374] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:30.134883 21080 raft_consensus.cc:729] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9ac302511e48239bf6fb88f3b3d1a5, State: Initialized, Role: FOLLOWER
I20260811 11:14:30.134963 21080 consensus_queue.cc:260] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.135201 21080 ts_tablet_manager.cc:1428] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 11:14:30.135289 20901 heartbeater.cc:503] Master 127.29.145.62:42067 was elected leader, sending a full tablet report...
W20260811 11:14:30.210492 20902 tablet.cc:2369] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:14:30.329011 20755 tablet.cc:2369] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:14:30.342116 21058 tablet.cc:2369] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:14:30.379251 21084 raft_consensus.cc:482] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:14:30.379562 21084 raft_consensus.cc:504] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.380188 21084 leader_election.cc:288] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41dc8722d78e460cb1c13c8e2e03f89e (127.29.145.3:33721), abb98461d728404dad141ae35c5d7ec1 (127.29.145.1:36087)
I20260811 11:14:30.384114 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.380850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:44415 (local address 127.29.145.1:36087)
0811 11:14:30.380937 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.380940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.381080 (+ 140us) server_negotiation.cc:407] Connection header received
0811 11:14:30.381153 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.381154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.381167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.381247 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:30.381429 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.382015 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383000 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.383597 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383679 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.383690 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.383939 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.384000 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:14:30.384246 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.380516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.3:33721 (local address 127.29.145.2:33585)
0811 11:14:30.380617 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.380634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.380724 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.380964 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.380966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.380968 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:30.381075 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.381078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.381960 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.381962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.382879 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.382883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.383825 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.384004 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.384010 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.384081 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.384147 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":1}
I20260811 11:14:30.384374 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.380631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.29.145.2:33585 (local address 127.29.145.3:33721)
0811 11:14:30.380730 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.380733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.380783 (+ 50us) server_negotiation.cc:407] Connection header received
0811 11:14:30.380845 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.380846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.380862 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.380933 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:14:30.381176 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.381803 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383012 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.383698 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383816 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.383836 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.384138 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.384223 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:14:30.384591 21085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.380717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:36087 (local address 127.29.145.2:44415)
0811 11:14:30.380994 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.381008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:30.381089 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:30.381251 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:30.381252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:30.381254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:14:30.381350 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.381352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.382105 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.382107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.382891 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:30.382893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.383731 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.383733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:30.383831 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.383837 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:14:30.384392 (+ 555us) client_negotiation.cc:240] Negotiation successful
0811 11:14:30.384513 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":157,"threads_started":1}
I20260811 11:14:30.384879 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce12312b52be4bd997d29fa81c4c1d30" candidate_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" is_pre_election: true
I20260811 11:14:30.385128 21004 raft_consensus.cc:2454] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9ac302511e48239bf6fb88f3b3d1a5 in term 0.
I20260811 11:14:30.385093 20701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce12312b52be4bd997d29fa81c4c1d30" candidate_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb98461d728404dad141ae35c5d7ec1" is_pre_election: true
I20260811 11:14:30.385282 20701 raft_consensus.cc:2454] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9ac302511e48239bf6fb88f3b3d1a5 in term 0.
I20260811 11:14:30.385447 20776 leader_election.cc:302] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41dc8722d78e460cb1c13c8e2e03f89e, 4c9ac302511e48239bf6fb88f3b3d1a5; no voters:
I20260811 11:14:30.385625 21084 raft_consensus.cc:2781] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:14:30.385699 21084 raft_consensus.cc:482] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:14:30.385747 21084 raft_consensus.cc:3037] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:30.386770 21084 raft_consensus.cc:504] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.386967 21084 leader_election.cc:288] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [CANDIDATE]: Term 1 election: Requested vote from peers 41dc8722d78e460cb1c13c8e2e03f89e (127.29.145.3:33721), abb98461d728404dad141ae35c5d7ec1 (127.29.145.1:36087)
I20260811 11:14:30.387180 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce12312b52be4bd997d29fa81c4c1d30" candidate_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41dc8722d78e460cb1c13c8e2e03f89e"
I20260811 11:14:30.387190 20701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce12312b52be4bd997d29fa81c4c1d30" candidate_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abb98461d728404dad141ae35c5d7ec1"
I20260811 11:14:30.387290 20701 raft_consensus.cc:3037] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:30.387290 21004 raft_consensus.cc:3037] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:30.388517 20701 raft_consensus.cc:2454] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9ac302511e48239bf6fb88f3b3d1a5 in term 1.
I20260811 11:14:30.388517 21004 raft_consensus.cc:2454] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9ac302511e48239bf6fb88f3b3d1a5 in term 1.
I20260811 11:14:30.388784 20776 leader_election.cc:302] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9ac302511e48239bf6fb88f3b3d1a5, abb98461d728404dad141ae35c5d7ec1; no voters:
I20260811 11:14:30.388923 21084 raft_consensus.cc:2781] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:30.389189 21084 raft_consensus.cc:686] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 1 LEADER]: Becoming Leader. State: Replica: 4c9ac302511e48239bf6fb88f3b3d1a5, State: Running, Role: LEADER
I20260811 11:14:30.389354 21084 consensus_queue.cc:237] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } }
I20260811 11:14:30.390959 20529 catalog_manager.cc:5495] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9ac302511e48239bf6fb88f3b3d1a5 (127.29.145.2). New cstate: current_term: 1 leader_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9ac302511e48239bf6fb88f3b3d1a5" member_type: VOTER last_known_addr { host: "127.29.145.2" port: 33133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 } health_report { overall_health: UNKNOWN } } }
I20260811 11:14:30.407531 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.404059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.29.145.2:33133)
0811 11:14:30.404172 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:30.404174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:30.404186 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:14:30.404240 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:30.404241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:30.404269 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:30.404361 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:30.404605 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.405230 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.406298 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:30.406583 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:30.406667 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:30.406901 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:30.407056 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:30.407171 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:30.407250 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 11:14:30.407276 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 11:14:30.410439 21004 raft_consensus.cc:1261] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 1 FOLLOWER]: Refusing update from remote peer 4c9ac302511e48239bf6fb88f3b3d1a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:30.410457 20701 raft_consensus.cc:1261] T ce12312b52be4bd997d29fa81c4c1d30 P abb98461d728404dad141ae35c5d7ec1 [term 1 FOLLOWER]: Refusing update from remote peer 4c9ac302511e48239bf6fb88f3b3d1a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:30.411032 21086 consensus_queue.cc:1035] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb98461d728404dad141ae35c5d7ec1" member_type: VOTER last_known_addr { host: "127.29.145.1" port: 36087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:30.411222 21084 consensus_queue.cc:1035] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41dc8722d78e460cb1c13c8e2e03f89e" member_type: VOTER last_known_addr { host: "127.29.145.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:30.415202 21090 mvcc.cc:205] Tried to move back new op lower bound from 7317286381196627968 to 7317286381115740160. Current Snapshot: MvccSnapshot[applied={T|T < 7317286381196627968 or (T in {7317286381196627968})}]
I20260811 11:14:30.417408 20529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:30.417567 20529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:14:30.418728 20529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f4656a1270d442681abee365cc969ed: Sending DeleteTablet for 3 replicas of tablet ce12312b52be4bd997d29fa81c4c1d30
I20260811 11:14:30.419272 20978 tablet_service.cc:1514] Processing DeleteTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:30 UTC) from {username='slave'} at 127.0.0.1:46442
I20260811 11:14:30.419327 20827 tablet_service.cc:1514] Processing DeleteTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:30 UTC) from {username='slave'} at 127.0.0.1:58536
I20260811 11:14:30.419711 21101 tablet_replica.cc:330] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: stopping tablet replica
I20260811 11:14:30.419752 20680 tablet_service.cc:1514] Processing DeleteTablet for tablet ce12312b52be4bd997d29fa81c4c1d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:14:30 UTC) from {username='slave'} at 127.0.0.1:37468
I20260811 11:14:30.419853 21101 raft_consensus.cc:2229] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:14:30.419975 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20605
I20260811 11:14:30.420018 21101 raft_consensus.cc:2258] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:30.420631 21102 tablet_replica.cc:330] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: stopping tablet replica
I20260811 11:14:30.422034 21102 raft_consensus.cc:2229] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:14:30.422173 21102 raft_consensus.cc:2258] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:14:30.423226 21102 ts_tablet_manager.cc:1905] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:30.424044 21101 ts_tablet_manager.cc:1905] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:14:30.425412 21102 ts_tablet_manager.cc:1918] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:30.425501 21102 log.cc:1196] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-1/wal/wals/ce12312b52be4bd997d29fa81c4c1d30
I20260811 11:14:30.425800 21102 ts_tablet_manager.cc:1939] T ce12312b52be4bd997d29fa81c4c1d30 P 4c9ac302511e48239bf6fb88f3b3d1a5: Deleting consensus metadata
I20260811 11:14:30.426412 20516 catalog_manager.cc:4841] TS 4c9ac302511e48239bf6fb88f3b3d1a5 (127.29.145.2:33133): tablet ce12312b52be4bd997d29fa81c4c1d30 (table test-table [id=fa2a70dfecbd481981e56b33b83bbcee]) successfully deleted
I20260811 11:14:30.426803 21101 ts_tablet_manager.cc:1918] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:14:30.426895 21101 log.cc:1196] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Deleting WAL directory at /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786446611237378-30276-0/minicluster-data/ts-2/wal/wals/ce12312b52be4bd997d29fa81c4c1d30
I20260811 11:14:30.427217 21101 ts_tablet_manager.cc:1939] T ce12312b52be4bd997d29fa81c4c1d30 P 41dc8722d78e460cb1c13c8e2e03f89e: Deleting consensus metadata
I20260811 11:14:30.427767 20515 catalog_manager.cc:4841] TS 41dc8722d78e460cb1c13c8e2e03f89e (127.29.145.3:33721): tablet ce12312b52be4bd997d29fa81c4c1d30 (table test-table [id=fa2a70dfecbd481981e56b33b83bbcee]) successfully deleted
W20260811 11:14:30.435561 20515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.29.145.1:36087 (error 108)
I20260811 11:14:30.435774 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20757
I20260811 11:14:30.436262 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:30.436114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.1:36087 (local address 127.0.0.1:37470)
0811 11:14:30.436174 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:30.436215 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.1:36087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 11:14:30.436398 20515 catalog_manager.cc:4584] TS abb98461d728404dad141ae35c5d7ec1 (127.29.145.1:36087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ce12312b52be4bd997d29fa81c4c1d30: Network error: Client connection negotiation failed: client connection to 127.29.145.1:36087: connect: Connection refused (error 111)
I20260811 11:14:30.448840 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20904
I20260811 11:14:30.461136 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 20496
2026-08-11T11:14:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (969 ms)
[----------] 6 tests from EncryptionPolicyTest (9434 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:30 dist-test-slave-69rg krb5kdc[21108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:30 dist-test-slave-69rg krb5kdc[21108](info): set up 2 sockets
Aug 11 11:14:30 dist-test-slave-69rg krb5kdc[21108](info): commencing operation
krb5kdc: starting...
W20260811 11:14:31.655793 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:31 dist-test-slave-69rg krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:31.787222 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:31.872206 21127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:31.872459 21127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:31.872498 21127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:31.873631 21127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:31.873703 21127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:31.873728 21127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:31.873755 21127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:31.873785 21127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:31.875308 21127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38529
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:31.875597 21127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:31.875833 21127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:31.880905 21138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:31.880882 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:31.880926 21135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:31.881163 21127 server_base.cc:1034] running on GCE node
I20260811 11:14:31.881464 21127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:31.881803 21127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:31.883008 21127 hybrid_clock.cc:648] HybridClock initialized: now 1786446871882994 us; error 31 us; skew 500 ppm
Aug 11 11:14:31 dist-test-slave-69rg krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:31.885680 21127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:31.886188 21127 webserver.cc:460] Webserver started at http://127.0.0.1:39847/ using document root <none> and password file <none>
I20260811 11:14:31.886396 21127 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:31.886438 21127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:31.886524 21127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:31.887540 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "e87f18ac64f642c383f54968d8222f68"
format_stamp: "Formatted at 2026-08-11 11:14:31 on dist-test-slave-69rg"
server_key: "13c33299d2f202844d82c1e881f39e00"
server_key_iv: "4c0cd47e24cd4ed7ef79cfedc0a280a9"
server_key_version: "encryptionkey@0"
I20260811 11:14:31.887848 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "e87f18ac64f642c383f54968d8222f68"
format_stamp: "Formatted at 2026-08-11 11:14:31 on dist-test-slave-69rg"
server_key: "13c33299d2f202844d82c1e881f39e00"
server_key_iv: "4c0cd47e24cd4ed7ef79cfedc0a280a9"
server_key_version: "encryptionkey@0"
I20260811 11:14:31.889186 21127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:31.890003 21145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:31.890225 21127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:31.890316 21127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "e87f18ac64f642c383f54968d8222f68"
format_stamp: "Formatted at 2026-08-11 11:14:31 on dist-test-slave-69rg"
server_key: "13c33299d2f202844d82c1e881f39e00"
server_key_iv: "4c0cd47e24cd4ed7ef79cfedc0a280a9"
server_key_version: "encryptionkey@0"
I20260811 11:14:31.890372 21127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:31.918977 21127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:31.921106 21127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:31.921227 21127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:31.925563 21127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38529
I20260811 11:14:31.925645 21209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38529 every 8 connection(s)
I20260811 11:14:31.926178 21127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:31.926928 21210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:31.929234 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68: Bootstrap starting.
I20260811 11:14:31.929908 21210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:31.930203 21210 log.cc:824] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:31.930928 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68: No bootstrap required, opened a new log
I20260811 11:14:31.932533 21210 raft_consensus.cc:348] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } }
I20260811 11:14:31.932698 21210 raft_consensus.cc:374] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:31.932780 21210 raft_consensus.cc:729] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87f18ac64f642c383f54968d8222f68, State: Initialized, Role: FOLLOWER
I20260811 11:14:31.932971 21210 consensus_queue.cc:260] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } }
I20260811 11:14:31.933077 21210 raft_consensus.cc:388] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:31.933121 21210 raft_consensus.cc:482] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:31.933176 21210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:31.933373 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21127
I20260811 11:14:31.933495 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:31.933712 30276 external_mini_cluster.cc:1428] Setting key 39e918b3f8d828ae67a8ebc2abd9b42a
I20260811 11:14:31.934382 21210 raft_consensus.cc:504] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } }
I20260811 11:14:31.934494 21210 leader_election.cc:302] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e87f18ac64f642c383f54968d8222f68; no voters:
I20260811 11:14:31.934695 21210 leader_election.cc:288] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:31.934719 21217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:31.934839 21217 raft_consensus.cc:686] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [term 1 LEADER]: Becoming Leader. State: Replica: e87f18ac64f642c383f54968d8222f68, State: Running, Role: LEADER
I20260811 11:14:31.934928 21217 consensus_queue.cc:237] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } }
I20260811 11:14:31.934973 21210 sys_catalog.cc:564] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:31.935308 21217 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e87f18ac64f642c383f54968d8222f68. Latest consensus state: current_term: 1 leader_uuid: "e87f18ac64f642c383f54968d8222f68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } } }
I20260811 11:14:31.935396 21217 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:31.935647 21218 sys_catalog.cc:455] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e87f18ac64f642c383f54968d8222f68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e87f18ac64f642c383f54968d8222f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38529 } } }
I20260811 11:14:31.935815 21218 sys_catalog.cc:458] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:31.935747 21223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:31.936550 21223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:31.938277 21223 catalog_manager.cc:1347] Generated new cluster ID: 7fc8847630784981a3827db278cf7a18
I20260811 11:14:31.938329 21223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:14:31 dist-test-slave-69rg krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:14:31.944919 21223 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:31.945628 21223 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:31.946172 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:31.934260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33389 (local address 127.0.0.1:38529)
0811 11:14:31.934477 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:31.934488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:31.934522 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:14:31.934663 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:31.934670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:31.934722 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:31.934873 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:31.935858 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:31.937271 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:31.938461 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:31.939047 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:31.939154 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:31.942045 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:31.942050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:31.942050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:31.942063 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:31.944222 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:31.944815 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:31.944819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:31.944819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:31.944933 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:31.945212 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:31.945213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:31.945214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:31.945491 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:31.945729 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:31.945855 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 11:14:31.945964 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":113,"threads_started":1}
I20260811 11:14:31.947266 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:31.951030 21223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e87f18ac64f642c383f54968d8222f68: Generated new TSK 0
I20260811 11:14:31.951212 21223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:31.956631 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:31.949026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55471 (local address 127.0.0.1:38529)
0811 11:14:31.949065 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:31.949066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:31.949987 (+ 921us) server_negotiation.cc:407] Connection header received
0811 11:14:31.950046 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:31.950047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:31.950057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:31.950118 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:31.950816 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:31.951623 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:31.952335 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:31.952703 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:31.952801 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:31.954059 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:31.954061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:31.954061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:31.954068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:31.955520 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:31.955875 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:31.955876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:31.955877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:31.955935 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:31.956142 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:31.956143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:31.956143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:31.956264 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:31.956372 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:31.956469 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 11:14:31.956534 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:14:31.958685 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21127
W20260811 11:14:31.968222 21105 connection.cc:535] client connection to 127.0.0.1:38529 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38529): Connection reset by peer (error 104)
2026-08-11T11:14:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:32 dist-test-slave-69rg krb5kdc[21248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:32 dist-test-slave-69rg krb5kdc[21248](info): set up 2 sockets
Aug 11 11:14:32 dist-test-slave-69rg krb5kdc[21248](info): commencing operation
krb5kdc: starting...
W20260811 11:14:33.084391 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:33 dist-test-slave-69rg krb5kdc[21248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:33.214562 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:33.299508 21265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:33.299715 21265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:33.299738 21265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:33.300864 21265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:33.300906 21265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:33.300921 21265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:33.300933 21265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:33.300946 21265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:33.302451 21265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:33.302706 21265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:33.302950 21265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:33.307991 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:33.307951 21271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:33.307961 21275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:33.308283 21265 server_base.cc:1034] running on GCE node
I20260811 11:14:33.308561 21265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:33.308843 21265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:33.309974 21265 hybrid_clock.cc:648] HybridClock initialized: now 1786446873309944 us; error 52 us; skew 500 ppm
Aug 11 11:14:33 dist-test-slave-69rg krb5kdc[21248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:33.312604 21265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:33.313050 21265 webserver.cc:460] Webserver started at http://127.0.0.1:43873/ using document root <none> and password file <none>
I20260811 11:14:33.313305 21265 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:33.313374 21265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:33.313525 21265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:33.314692 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "96aa7394e55a470c9c6bed92f3c216dc"
format_stamp: "Formatted at 2026-08-11 11:14:33 on dist-test-slave-69rg"
server_key: "5ea7bedb302ee9baabe48b22b1cc9fcd"
server_key_iv: "198f4d2848e8628d438d33a03ca324d2"
server_key_version: "encryptionkey@0"
I20260811 11:14:33.315064 21265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "96aa7394e55a470c9c6bed92f3c216dc"
format_stamp: "Formatted at 2026-08-11 11:14:33 on dist-test-slave-69rg"
server_key: "5ea7bedb302ee9baabe48b22b1cc9fcd"
server_key_iv: "198f4d2848e8628d438d33a03ca324d2"
server_key_version: "encryptionkey@0"
I20260811 11:14:33.316478 21265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:14:33.317330 21284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:33.317535 21265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:33.317652 21265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "96aa7394e55a470c9c6bed92f3c216dc"
format_stamp: "Formatted at 2026-08-11 11:14:33 on dist-test-slave-69rg"
server_key: "5ea7bedb302ee9baabe48b22b1cc9fcd"
server_key_iv: "198f4d2848e8628d438d33a03ca324d2"
server_key_version: "encryptionkey@0"
I20260811 11:14:33.317734 21265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:33.337554 21265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:33.340138 21265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:33.340278 21265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:33.344483 21265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38011
I20260811 11:14:33.344573 21343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38011 every 8 connection(s)
I20260811 11:14:33.344915 21265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:33.345978 21344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:33.348610 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc: Bootstrap starting.
I20260811 11:14:33.349282 21344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:33.349610 21344 log.cc:824] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:33.350533 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc: No bootstrap required, opened a new log
I20260811 11:14:33.350572 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21265
I20260811 11:14:33.350700 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:33.350904 30276 external_mini_cluster.cc:1428] Setting key 748d94f11a04c39081cea1089be6b5e7
I20260811 11:14:33.352605 21344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } }
I20260811 11:14:33.352782 21344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:33.352854 21344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96aa7394e55a470c9c6bed92f3c216dc, State: Initialized, Role: FOLLOWER
I20260811 11:14:33.353029 21344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } }
I20260811 11:14:33.353140 21344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:33.353192 21344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:33.353248 21344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:33.353911 21344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } }
I20260811 11:14:33.354031 21344 leader_election.cc:302] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96aa7394e55a470c9c6bed92f3c216dc; no voters:
I20260811 11:14:33.354220 21344 leader_election.cc:288] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:33.354260 21349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:33.354401 21349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [term 1 LEADER]: Becoming Leader. State: Replica: 96aa7394e55a470c9c6bed92f3c216dc, State: Running, Role: LEADER
I20260811 11:14:33.354508 21349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } }
I20260811 11:14:33.354512 21344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:33.355126 21351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96aa7394e55a470c9c6bed92f3c216dc. Latest consensus state: current_term: 1 leader_uuid: "96aa7394e55a470c9c6bed92f3c216dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } } }
I20260811 11:14:33.355286 21351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:33.355120 21350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96aa7394e55a470c9c6bed92f3c216dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96aa7394e55a470c9c6bed92f3c216dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38011 } } }
I20260811 11:14:33.355563 21350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:33.356280 21359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:33.356805 21359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:33.358649 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:33.351485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44847 (local address 127.0.0.1:38011)
0811 11:14:33.351679 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:33.351687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:33.351712 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:14:33.352604 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:33.352615 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:33.352673 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:33.352799 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:33.353153 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:33.355105 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:33.356322 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:33.356884 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:33.357024 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:33.357812 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:33.357836 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:33.357837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:33.358108 (+ 271us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:33.358149 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:33.358164 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:33.358271 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:33.358404 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 11:14:33.358435 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260811 11:14:33.358783 21359 catalog_manager.cc:1347] Generated new cluster ID: bc4dd2c1496a4ecd87673d78c0a2d569
I20260811 11:14:33.358829 21359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:33.359711 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:33.364703 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:33.361206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55527 (local address 127.0.0.1:38011)
0811 11:14:33.361251 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:33.361253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:33.361311 (+ 58us) server_negotiation.cc:407] Connection header received
0811 11:14:33.361370 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:33.361370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:33.361380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:33.361452 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:33.361584 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:33.362359 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:33.363334 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:33.363665 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:33.363742 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:33.364438 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:33.364439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:33.364440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:33.364444 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:33.364460 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:33.364465 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:33.364526 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:33.364617 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 11:14:33.364631 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:14:33.388360 21359 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:33.389130 21359 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:33.401022 21359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96aa7394e55a470c9c6bed92f3c216dc: Generated new TSK 0
I20260811 11:14:33.401259 21359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:33.404544 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21265
W20260811 11:14:33.414723 21246 connection.cc:535] client connection to 127.0.0.1:38011 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38011): Connection reset by peer (error 104)
2026-08-11T11:14:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:33 dist-test-slave-69rg krb5kdc[21379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:33 dist-test-slave-69rg krb5kdc[21379](info): set up 2 sockets
Aug 11 11:14:33 dist-test-slave-69rg krb5kdc[21379](info): commencing operation
krb5kdc: starting...
W20260811 11:14:34.523161 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:34 dist-test-slave-69rg krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:34.653142 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:34.736865 21393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:34.737118 21393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:34.737156 21393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:34.738349 21393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:34.738421 21393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:34.738445 21393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:34.738467 21393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:34.738494 21393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:34.739966 21393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:34.740247 21393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:34.740480 21393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:34.745759 21402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:34.745725 21405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:34.745769 21403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:34.746129 21393 server_base.cc:1034] running on GCE node
I20260811 11:14:34.746366 21393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:34.746676 21393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:34.747838 21393 hybrid_clock.cc:648] HybridClock initialized: now 1786446874747827 us; error 28 us; skew 500 ppm
Aug 11 11:14:34 dist-test-slave-69rg krb5kdc[21379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:34.769618 21393 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:34.770228 21393 webserver.cc:460] Webserver started at http://127.0.0.1:39541/ using document root <none> and password file <none>
I20260811 11:14:34.770476 21393 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:34.770546 21393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:34.770704 21393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:34.771827 21393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "591a2304b34a4e4387725348e38e1a59"
format_stamp: "Formatted at 2026-08-11 11:14:34 on dist-test-slave-69rg"
server_key: "f5c4b1e436ec65c5ab94496f003802e0"
server_key_iv: "184af2b55a8c478ebf67baf98a165cc7"
server_key_version: "encryptionkey@0"
I20260811 11:14:34.772215 21393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "591a2304b34a4e4387725348e38e1a59"
format_stamp: "Formatted at 2026-08-11 11:14:34 on dist-test-slave-69rg"
server_key: "f5c4b1e436ec65c5ab94496f003802e0"
server_key_iv: "184af2b55a8c478ebf67baf98a165cc7"
server_key_version: "encryptionkey@0"
I20260811 11:14:34.773641 21393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:34.774570 21412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:34.774760 21393 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:34.774873 21393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "591a2304b34a4e4387725348e38e1a59"
format_stamp: "Formatted at 2026-08-11 11:14:34 on dist-test-slave-69rg"
server_key: "f5c4b1e436ec65c5ab94496f003802e0"
server_key_iv: "184af2b55a8c478ebf67baf98a165cc7"
server_key_version: "encryptionkey@0"
I20260811 11:14:34.774948 21393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:34.778437 21393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:34.780423 21393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:34.780597 21393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:34.784971 21393 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42251
I20260811 11:14:34.785012 21473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42251 every 8 connection(s)
I20260811 11:14:34.785373 21393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:34.786473 21474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:34.788713 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21393
I20260811 11:14:34.788757 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: Bootstrap starting.
I20260811 11:14:34.788854 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:34.789043 30276 external_mini_cluster.cc:1428] Setting key dfee9bce1cc64fef81be63452a1228ca
I20260811 11:14:34.789530 21474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:34.790040 21474 log.cc:824] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:34.790822 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: No bootstrap required, opened a new log
I20260811 11:14:34.791831 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:34.789548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57209 (local address 127.0.0.1:42251)
0811 11:14:34.789766 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:34.789777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:34.789857 (+ 80us) server_negotiation.cc:407] Connection header received
0811 11:14:34.790494 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:34.790503 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:34.790538 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:34.790688 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:34.791348 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:34.791352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:34.791353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:34.791358 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:34.791402 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:34.791416 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:34.791512 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:34.791592 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 11:14:34.791634 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260811 11:14:34.792976 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:34.793489 21474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } }
I20260811 11:14:34.793622 21474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:34.793643 21474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591a2304b34a4e4387725348e38e1a59, State: Initialized, Role: FOLLOWER
I20260811 11:14:34.793784 21474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } }
I20260811 11:14:34.793921 21474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:34.793984 21474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:34.794085 21474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:34.794763 21474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } }
I20260811 11:14:34.794874 21474 leader_election.cc:302] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 591a2304b34a4e4387725348e38e1a59; no voters:
I20260811 11:14:34.795038 21474 leader_election.cc:288] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:34.795107 21486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:34.795255 21486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [term 1 LEADER]: Becoming Leader. State: Replica: 591a2304b34a4e4387725348e38e1a59, State: Running, Role: LEADER
I20260811 11:14:34.795352 21474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:34.795400 21486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } }
I20260811 11:14:34.795617 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:34.794502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36277 (local address 127.0.0.1:42251)
0811 11:14:34.794543 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:34.794544 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:34.794609 (+ 65us) server_negotiation.cc:407] Connection header received
0811 11:14:34.794673 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:34.794674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:34.794679 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:34.794739 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:34.795383 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:34.795385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:34.795385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:34.795390 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:34.795403 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:34.795408 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:34.795439 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:34.795552 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 11:14:34.795565 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:14:34.795850 21487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "591a2304b34a4e4387725348e38e1a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } } }
I20260811 11:14:34.795894 21488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 591a2304b34a4e4387725348e38e1a59. Latest consensus state: current_term: 1 leader_uuid: "591a2304b34a4e4387725348e38e1a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591a2304b34a4e4387725348e38e1a59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42251 } } }
I20260811 11:14:34.795948 21488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:34.795948 21487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59 [sys.catalog]: This master's current role is: LEADER
W20260811 11:14:34.797231 21504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:34.797281 21504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:34.797349 21497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:34.797505 21497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:34.798961 21497 catalog_manager.cc:1347] Generated new cluster ID: b15a217d8f6f46538692406c0e4630e9
I20260811 11:14:34.799014 21497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:34.805387 21497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:34.805562 21497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:34.812114 21497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 591a2304b34a4e4387725348e38e1a59: Generated new TSK 0
I20260811 11:14:34.812300 21497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:34.815510 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21393
2026-08-11T11:14:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:34 dist-test-slave-69rg krb5kdc[21511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:34 dist-test-slave-69rg krb5kdc[21511](info): set up 2 sockets
Aug 11 11:14:34 dist-test-slave-69rg krb5kdc[21511](info): commencing operation
krb5kdc: starting...
W20260811 11:14:35.948143 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:36.081357 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:36.166157 21528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:36.166378 21528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:36.166415 21528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:36.167531 21528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:36.167596 21528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:36.167621 21528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:36.167661 21528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:36.167687 21528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:36.169149 21528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:36.169420 21528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:36.169646 21528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:36.174697 21535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:36.174705 21537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:36.174821 21534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:36.175019 21528 server_base.cc:1034] running on GCE node
I20260811 11:14:36.175403 21528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:36.175652 21528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:36.176828 21528 hybrid_clock.cc:648] HybridClock initialized: now 1786446876176828 us; error 40 us; skew 500 ppm
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446876, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:36.179545 21528 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:36.179935 21528 webserver.cc:460] Webserver started at http://127.0.0.1:39931/ using document root <none> and password file <none>
I20260811 11:14:36.180147 21528 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:36.180197 21528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:36.180284 21528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:36.181301 21528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "d829b75d096b42d08f6f00b13fde9598"
format_stamp: "Formatted at 2026-08-11 11:14:36 on dist-test-slave-69rg"
server_key: "c464ad64fe91b84ac51ce6a9120cee73"
server_key_iv: "14a54503f1a0090a08f5a512813bb992"
server_key_version: "encryptionkey@0"
I20260811 11:14:36.181614 21528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "d829b75d096b42d08f6f00b13fde9598"
format_stamp: "Formatted at 2026-08-11 11:14:36 on dist-test-slave-69rg"
server_key: "c464ad64fe91b84ac51ce6a9120cee73"
server_key_iv: "14a54503f1a0090a08f5a512813bb992"
server_key_version: "encryptionkey@0"
I20260811 11:14:36.183101 21528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:36.183866 21546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:36.184077 21528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:36.184165 21528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "d829b75d096b42d08f6f00b13fde9598"
format_stamp: "Formatted at 2026-08-11 11:14:36 on dist-test-slave-69rg"
server_key: "c464ad64fe91b84ac51ce6a9120cee73"
server_key_iv: "14a54503f1a0090a08f5a512813bb992"
server_key_version: "encryptionkey@0"
I20260811 11:14:36.184224 21528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:36.228130 21528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:36.230723 21528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:36.230854 21528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:36.234848 21528 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44735
I20260811 11:14:36.234942 21613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44735 every 8 connection(s)
I20260811 11:14:36.235283 21528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:36.236267 21614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:36.237362 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21528
I20260811 11:14:36.237494 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:36.237720 30276 external_mini_cluster.cc:1428] Setting key ee4e874ed4bb9260ef36cc833826c459
I20260811 11:14:36.239225 21614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598: Bootstrap starting.
I20260811 11:14:36.239838 21614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:36.240146 21614 log.cc:824] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:36.240883 21614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598: No bootstrap required, opened a new log
I20260811 11:14:36.243482 21614 raft_consensus.cc:348] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260811 11:14:36.243695 21614 raft_consensus.cc:374] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:36.243737 21614 raft_consensus.cc:729] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d829b75d096b42d08f6f00b13fde9598, State: Initialized, Role: FOLLOWER
I20260811 11:14:36.243877 21614 consensus_queue.cc:260] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260811 11:14:36.243968 21614 raft_consensus.cc:388] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:36.243999 21614 raft_consensus.cc:482] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:36.244046 21614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:36.244697 21614 raft_consensus.cc:504] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:14:36.244791 21614 leader_election.cc:302] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d829b75d096b42d08f6f00b13fde9598; no voters:
I20260811 11:14:36.245031 21614 leader_election.cc:288] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:36.245057 21620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:36.245185 21620 raft_consensus.cc:686] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [term 1 LEADER]: Becoming Leader. State: Replica: d829b75d096b42d08f6f00b13fde9598, State: Running, Role: LEADER
I20260811 11:14:36.245285 21620 consensus_queue.cc:237] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } }
I20260811 11:14:36.245321 21614 sys_catalog.cc:564] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:36.246001 21622 sys_catalog.cc:455] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d829b75d096b42d08f6f00b13fde9598. Latest consensus state: current_term: 1 leader_uuid: "d829b75d096b42d08f6f00b13fde9598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } } }
I20260811 11:14:36.245995 21621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d829b75d096b42d08f6f00b13fde9598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d829b75d096b42d08f6f00b13fde9598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44735 } } }
I20260811 11:14:36.246114 21621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:36.246273 21622 sys_catalog.cc:458] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:36.246402 21625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:36.246917 21625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:36.248884 21625 catalog_manager.cc:1347] Generated new cluster ID: 3aa09bd291ed41a68e7dbffa12557ecb
I20260811 11:14:36.248936 21625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:36.251307 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:36.238305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60701 (local address 127.0.0.1:44735)
0811 11:14:36.238513 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:36.238523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:36.238561 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:14:36.239066 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:36.239073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:36.239118 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:36.239283 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:36.239946 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:36.240935 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:36.242062 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:36.242498 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:36.242583 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:36.245844 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:36.245851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:36.245851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:36.245866 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:36.249657 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:36.250116 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:36.250120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:36.250120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:36.250234 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:36.250460 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:36.250461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:36.250461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:36.250718 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:36.250901 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:36.250971 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 11:14:36.251124 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260811 11:14:36.254555 21625 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:36.254993 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:36.255237 21625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:36.263758 21625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d829b75d096b42d08f6f00b13fde9598: Generated new TSK 0
I20260811 11:14:36.263976 21625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:36.273200 21618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:36.266227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40635 (local address 127.0.0.1:44735)
0811 11:14:36.266290 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:36.266292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:36.266345 (+ 53us) server_negotiation.cc:407] Connection header received
0811 11:14:36.266418 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:36.266419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:36.266428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:36.266484 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:36.267197 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:36.267726 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:36.268834 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:36.269037 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:36.269094 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:36.270623 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:36.270625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:36.270625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:36.270631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:36.272102 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:36.272507 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:36.272508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:36.272508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:36.272557 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:36.272763 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:36.272764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:36.272764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:36.272860 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:36.272933 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:36.273068 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 11:14:36.273136 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:14:36.275367 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21528
2026-08-11T11:14:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21648](info): set up 2 sockets
Aug 11 11:14:36 dist-test-slave-69rg krb5kdc[21648](info): commencing operation
krb5kdc: starting...
W20260811 11:14:37.401407 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:37 dist-test-slave-69rg krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:37.529395 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:37.614480 21662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:37.614722 21662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:37.614764 21662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:37.615898 21662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:37.615968 21662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:37.615993 21662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:37.616020 21662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:37.616051 21662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:37.617514 21662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:37.617801 21662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:37.618050 21662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:37.623035 21667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:37.623047 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:37.623123 21662 server_base.cc:1034] running on GCE node
W20260811 11:14:37.623052 21672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:37.623512 21662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:37.623816 21662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:37.624992 21662 hybrid_clock.cc:648] HybridClock initialized: now 1786446877624972 us; error 36 us; skew 500 ppm
Aug 11 11:14:37 dist-test-slave-69rg krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:37.627734 21662 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:37.628180 21662 webserver.cc:460] Webserver started at http://127.0.0.1:45269/ using document root <none> and password file <none>
I20260811 11:14:37.628448 21662 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:37.628523 21662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:37.628684 21662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:37.629768 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "fa17fb31af57451fbe952582a0495ba8"
format_stamp: "Formatted at 2026-08-11 11:14:37 on dist-test-slave-69rg"
server_key: "fd6a255bb75addfc7cbebd22bd592796"
server_key_iv: "0d86bf7d9c2b9b017170ead27b6a4774"
server_key_version: "encryptionkey@0"
I20260811 11:14:37.630215 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "fa17fb31af57451fbe952582a0495ba8"
format_stamp: "Formatted at 2026-08-11 11:14:37 on dist-test-slave-69rg"
server_key: "fd6a255bb75addfc7cbebd22bd592796"
server_key_iv: "0d86bf7d9c2b9b017170ead27b6a4774"
server_key_version: "encryptionkey@0"
I20260811 11:14:37.631570 21662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:37.632359 21679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:37.632542 21662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:37.632655 21662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "fa17fb31af57451fbe952582a0495ba8"
format_stamp: "Formatted at 2026-08-11 11:14:37 on dist-test-slave-69rg"
server_key: "fd6a255bb75addfc7cbebd22bd592796"
server_key_iv: "0d86bf7d9c2b9b017170ead27b6a4774"
server_key_version: "encryptionkey@0"
I20260811 11:14:37.632737 21662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:37.647374 21662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:37.649781 21662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:37.649972 21662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:37.654245 21662 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42555
I20260811 11:14:37.654287 21736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42555 every 8 connection(s)
I20260811 11:14:37.654726 21662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:37.655695 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:37.658203 21738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8: Bootstrap starting.
I20260811 11:14:37.658746 21738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:37.659003 21738 log.cc:824] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:37.659708 21738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8: No bootstrap required, opened a new log
I20260811 11:14:37.661309 21738 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } }
I20260811 11:14:37.661446 21738 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:37.661468 21738 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa17fb31af57451fbe952582a0495ba8, State: Initialized, Role: FOLLOWER
I20260811 11:14:37.661607 21738 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } }
I20260811 11:14:37.661693 21738 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:37.661717 21738 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:37.661746 21738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:37.662312 21738 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } }
I20260811 11:14:37.662401 21738 leader_election.cc:302] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa17fb31af57451fbe952582a0495ba8; no voters:
I20260811 11:14:37.662528 21738 leader_election.cc:288] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:37.662613 21742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:37.662817 21742 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [term 1 LEADER]: Becoming Leader. State: Replica: fa17fb31af57451fbe952582a0495ba8, State: Running, Role: LEADER
I20260811 11:14:37.662851 21738 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:37.662909 21742 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } }
I20260811 11:14:37.663367 21744 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa17fb31af57451fbe952582a0495ba8. Latest consensus state: current_term: 1 leader_uuid: "fa17fb31af57451fbe952582a0495ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } } }
I20260811 11:14:37.663463 21744 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:37.663465 21743 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa17fb31af57451fbe952582a0495ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa17fb31af57451fbe952582a0495ba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42555 } } }
I20260811 11:14:37.663592 21743 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:37.663729 21750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:37.664294 21750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:37.664870 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21662
I20260811 11:14:37.664984 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:37.665202 30276 external_mini_cluster.cc:1428] Setting key d7400f719d70f7d65694970897730dbc
I20260811 11:14:37.666188 21750 catalog_manager.cc:1347] Generated new cluster ID: 32f3b0893731423aa553f05732531e12
I20260811 11:14:37.666245 21750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:37.675709 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:37.665877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38021 (local address 127.0.0.1:42555)
0811 11:14:37.669959 (+ 4082us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:37.669968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:37.670005 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:14:37.670146 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:37.670153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:37.670210 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:37.670362 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:37.670614 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:37.672462 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:37.673516 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:37.674162 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:37.674272 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:37.674987 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:37.674991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:37.674991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:37.675219 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:37.675258 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:37.675272 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:37.675343 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:37.675411 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 11:14:37.675472 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4011,"thread_start_us":75,"threads_started":1}
I20260811 11:14:37.676641 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:37.678211 21750 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:37.678886 21750 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:37.684616 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:37.678178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42451 (local address 127.0.0.1:42555)
0811 11:14:37.681904 (+ 3726us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:37.681906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:37.681915 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:37.681941 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:37.681941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:37.681951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:37.682023 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:37.682155 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:37.682641 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:37.683389 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:37.683613 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:37.683692 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:37.684374 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:37.684376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:37.684376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:37.684382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:37.684397 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:37.684401 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:37.684448 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:37.684515 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 11:14:37.684538 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3704}
I20260811 11:14:37.693233 21750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa17fb31af57451fbe952582a0495ba8: Generated new TSK 0
I20260811 11:14:37.693413 21750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:37.697235 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21662
2026-08-11T11:14:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:37 dist-test-slave-69rg krb5kdc[21773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:37 dist-test-slave-69rg krb5kdc[21773](info): set up 2 sockets
Aug 11 11:14:37 dist-test-slave-69rg krb5kdc[21773](info): commencing operation
krb5kdc: starting...
W20260811 11:14:38.824987 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:38 dist-test-slave-69rg krb5kdc[21773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:38.954324 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46397
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:39.041420 21789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:39.041672 21789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:39.041711 21789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:39.042865 21789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:39.042938 21789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:39.042964 21789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:39.042999 21789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:39.043025 21789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:39.044519 21789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:39.044813 21789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:39.045063 21789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:39.050347 21797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:39.050390 21795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:39.050346 21794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:39.050565 21789 server_base.cc:1034] running on GCE node
I20260811 11:14:39.050848 21789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:39.051184 21789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:39.052377 21789 hybrid_clock.cc:648] HybridClock initialized: now 1786446879052333 us; error 61 us; skew 500 ppm
Aug 11 11:14:39 dist-test-slave-69rg krb5kdc[21773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:39.055190 21789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:39.055591 21789 webserver.cc:460] Webserver started at http://127.0.0.1:34089/ using document root <none> and password file <none>
I20260811 11:14:39.055831 21789 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:39.055878 21789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:39.056005 21789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:39.057082 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "9084eab73e44429abb53e3cf38d0421e"
format_stamp: "Formatted at 2026-08-11 11:14:39 on dist-test-slave-69rg"
server_key: "845bf0d264ff2769e56a71505603be7c"
server_key_iv: "429e3040241744e66f19cd6fad353e7f"
server_key_version: "encryptionkey@0"
I20260811 11:14:39.057449 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "9084eab73e44429abb53e3cf38d0421e"
format_stamp: "Formatted at 2026-08-11 11:14:39 on dist-test-slave-69rg"
server_key: "845bf0d264ff2769e56a71505603be7c"
server_key_iv: "429e3040241744e66f19cd6fad353e7f"
server_key_version: "encryptionkey@0"
I20260811 11:14:39.059118 21789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:39.059985 21804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:39.060209 21789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:39.060316 21789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "9084eab73e44429abb53e3cf38d0421e"
format_stamp: "Formatted at 2026-08-11 11:14:39 on dist-test-slave-69rg"
server_key: "845bf0d264ff2769e56a71505603be7c"
server_key_iv: "429e3040241744e66f19cd6fad353e7f"
server_key_version: "encryptionkey@0"
I20260811 11:14:39.060398 21789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:39.063800 21789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:39.066011 21789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:39.066174 21789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:39.070437 21789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46397
I20260811 11:14:39.070492 21864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46397 every 8 connection(s)
I20260811 11:14:39.070852 21789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:39.071965 21865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:39.074855 21865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e: Bootstrap starting.
I20260811 11:14:39.075562 21865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:39.075914 21865 log.cc:824] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:39.076778 21865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e: No bootstrap required, opened a new log
I20260811 11:14:39.078791 21865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } }
I20260811 11:14:39.078951 21865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:39.078976 21865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9084eab73e44429abb53e3cf38d0421e, State: Initialized, Role: FOLLOWER
I20260811 11:14:39.079241 21865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } }
I20260811 11:14:39.079339 21865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:39.079363 21865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:39.079419 21865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:39.080042 21865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } }
I20260811 11:14:39.080088 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21789
I20260811 11:14:39.080137 21865 leader_election.cc:302] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9084eab73e44429abb53e3cf38d0421e; no voters:
I20260811 11:14:39.080260 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:39.080348 21865 leader_election.cc:288] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:39.080374 21869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:39.080502 21869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [term 1 LEADER]: Becoming Leader. State: Replica: 9084eab73e44429abb53e3cf38d0421e, State: Running, Role: LEADER
I20260811 11:14:39.080505 30276 external_mini_cluster.cc:1428] Setting key ae71daf84ed50d43cf405b7a7c299456
I20260811 11:14:39.080657 21865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:39.080663 21869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } }
I20260811 11:14:39.081161 21871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9084eab73e44429abb53e3cf38d0421e. Latest consensus state: current_term: 1 leader_uuid: "9084eab73e44429abb53e3cf38d0421e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } } }
I20260811 11:14:39.081247 21871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:39.081425 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9084eab73e44429abb53e3cf38d0421e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084eab73e44429abb53e3cf38d0421e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46397 } } }
I20260811 11:14:39.081506 21875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:39.081532 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:39.082149 21875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:39.083942 21875 catalog_manager.cc:1347] Generated new cluster ID: 7cf46788f4bc46f38c8df8e2505f7037
I20260811 11:14:39.083999 21875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:39.084137 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:39.080995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42991 (local address 127.0.0.1:46397)
0811 11:14:39.081995 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:39.082006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:39.082053 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:14:39.082186 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:39.082194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:39.082229 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:39.082716 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:39.083523 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:39.083526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:39.083526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:39.083531 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:39.083577 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:39.083598 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:39.083683 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:39.083762 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 11:14:39.083900 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":69,"threads_started":1}
I20260811 11:14:39.085098 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:39.087796 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:39.086630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48493 (local address 127.0.0.1:46397)
0811 11:14:39.086663 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:39.086664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:39.086863 (+ 199us) server_negotiation.cc:407] Connection header received
0811 11:14:39.086941 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:39.086942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:39.086946 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:39.086971 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:39.087586 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:39.087588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:39.087588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:39.087592 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:39.087605 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:39.087609 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:39.087637 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:39.087727 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 11:14:39.087739 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 11:14:39.099287 21875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:39.099471 21875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:39.102329 21875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9084eab73e44429abb53e3cf38d0421e: Generated new TSK 0
I20260811 11:14:39.102526 21875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:39.108338 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21789
2026-08-11T11:14:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:39 dist-test-slave-69rg krb5kdc[21899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:39 dist-test-slave-69rg krb5kdc[21899](info): set up 2 sockets
Aug 11 11:14:39 dist-test-slave-69rg krb5kdc[21899](info): commencing operation
krb5kdc: starting...
W20260811 11:14:40.224565 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[21899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:40.361930 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:40.447953 21915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:40.448166 21915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:40.448189 21915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:40.449505 21915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:40.449553 21915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:40.449568 21915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:40.449581 21915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:40.449594 21915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:40.451107 21915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:40.451314 21915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:40.451480 21915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:40.456688 21931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:40.456738 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:40.456718 21926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:40.457085 21915 server_base.cc:1034] running on GCE node
I20260811 11:14:40.457382 21915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:40.457731 21915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:40.458906 21915 hybrid_clock.cc:648] HybridClock initialized: now 1786446880458896 us; error 71 us; skew 500 ppm
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[21899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446880, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:40.466286 21915 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:40.466722 21915 webserver.cc:460] Webserver started at http://127.0.0.1:41837/ using document root <none> and password file <none>
I20260811 11:14:40.466922 21915 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:40.466962 21915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:40.467051 21915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:40.468307 21915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "8c86215a43994f6f8978e6ade2d42f01"
format_stamp: "Formatted at 2026-08-11 11:14:40 on dist-test-slave-69rg"
server_key: "bed49b1d60bd7e93cb26879e34aa005d"
server_key_iv: "e3663e9bf3c577c727cce58894c9f3f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:40.468622 21915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "8c86215a43994f6f8978e6ade2d42f01"
format_stamp: "Formatted at 2026-08-11 11:14:40 on dist-test-slave-69rg"
server_key: "bed49b1d60bd7e93cb26879e34aa005d"
server_key_iv: "e3663e9bf3c577c727cce58894c9f3f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:40.469966 21915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:40.470716 21938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:40.470896 21915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:14:40.471006 21915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "8c86215a43994f6f8978e6ade2d42f01"
format_stamp: "Formatted at 2026-08-11 11:14:40 on dist-test-slave-69rg"
server_key: "bed49b1d60bd7e93cb26879e34aa005d"
server_key_iv: "e3663e9bf3c577c727cce58894c9f3f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:40.471084 21915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:40.484149 21915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:40.486176 21915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:40.486346 21915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:40.490622 21915 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34911
I20260811 11:14:40.490656 21996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34911 every 8 connection(s)
I20260811 11:14:40.491149 21915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:40.492426 21997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:40.496470 21997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01: Bootstrap starting.
I20260811 11:14:40.497321 21997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:40.497709 21997 log.cc:824] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:40.497805 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 21915
I20260811 11:14:40.497957 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:40.498168 30276 external_mini_cluster.cc:1428] Setting key 94feb1374a9754b9e10cadb41e802a77
I20260811 11:14:40.498670 21997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01: No bootstrap required, opened a new log
I20260811 11:14:40.501368 21997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260811 11:14:40.501560 21997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:40.501602 21997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c86215a43994f6f8978e6ade2d42f01, State: Initialized, Role: FOLLOWER
I20260811 11:14:40.501753 21997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260811 11:14:40.501878 21997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:40.501929 21997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:40.501991 21997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:40.502766 21997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260811 11:14:40.502887 21997 leader_election.cc:302] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c86215a43994f6f8978e6ade2d42f01; no voters:
I20260811 11:14:40.503047 21997 leader_election.cc:288] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:40.503104 22003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:40.503297 22003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [term 1 LEADER]: Becoming Leader. State: Replica: 8c86215a43994f6f8978e6ade2d42f01, State: Running, Role: LEADER
I20260811 11:14:40.503330 21997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:40.503391 22003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260811 11:14:40.503712 22003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c86215a43994f6f8978e6ade2d42f01. Latest consensus state: current_term: 1 leader_uuid: "8c86215a43994f6f8978e6ade2d42f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } } }
I20260811 11:14:40.503803 22003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:40.504045 22010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:40.504777 22005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c86215a43994f6f8978e6ade2d42f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c86215a43994f6f8978e6ade2d42f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } } }
I20260811 11:14:40.504877 22005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:40.506089 22010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:40.507877 22010 catalog_manager.cc:1347] Generated new cluster ID: ba98743920a54c8cb0d8efd60158d8fb
I20260811 11:14:40.507933 22010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[21899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:14:40.512430 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:40.498720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:41305 (local address 127.0.0.1:34911)
0811 11:14:40.498911 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:40.498920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:40.498958 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:14:40.500489 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:40.500501 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:40.500576 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:40.500751 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:40.501560 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:40.503601 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:40.504563 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:40.505292 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:40.505415 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:40.508766 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:40.508771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:40.508771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:40.508784 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:40.510671 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:40.511138 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:40.511142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:40.511142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:40.511239 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:40.511503 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:40.511505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:40.511505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:40.511787 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:40.511999 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:40.512095 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 11:14:40.512221 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260811 11:14:40.513568 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:40.514835 22010 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:40.515497 22010 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:40.520640 22010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c86215a43994f6f8978e6ade2d42f01: Generated new TSK 0
I20260811 11:14:40.520826 22010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:40.522410 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:40.515185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:53141 (local address 127.0.0.1:34911)
0811 11:14:40.515231 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:40.515233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:40.515356 (+ 123us) server_negotiation.cc:407] Connection header received
0811 11:14:40.515438 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:40.515439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:40.515494 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:40.515571 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:40.516187 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:40.516756 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:40.517545 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:40.517807 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:40.517931 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:40.519339 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:40.519341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:40.519341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:40.519348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:40.521073 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:40.521483 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:40.521484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:40.521485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:40.521548 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:40.521806 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:40.521807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:40.521808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:40.521949 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:40.522078 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:40.522235 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 11:14:40.522308 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:14:40.524545 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 21915
W20260811 11:14:40.532050 21897 connection.cc:535] client connection to 127.0.0.1:34911 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34911): Connection reset by peer (error 104)
2026-08-11T11:14:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[22034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[22034](info): set up 2 sockets
Aug 11 11:14:40 dist-test-slave-69rg krb5kdc[22034](info): commencing operation
krb5kdc: starting...
W20260811 11:14:41.645006 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:41 dist-test-slave-69rg krb5kdc[22034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:41.773645 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:41.857265 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:41.857514 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:41.857553 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:41.858726 22050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:41.858799 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:41.858824 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:41.858853 22050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:41.858886 22050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:41.860354 22050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:41.860641 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:41.860877 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:41.866403 22060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:41.866384 22055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:41.866437 22056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:41.866597 22050 server_base.cc:1034] running on GCE node
I20260811 11:14:41.866920 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:41.867172 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:41.868348 22050 hybrid_clock.cc:648] HybridClock initialized: now 1786446881868334 us; error 47 us; skew 500 ppm
Aug 11 11:14:41 dist-test-slave-69rg krb5kdc[22034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:41.870926 22050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:41.871428 22050 webserver.cc:460] Webserver started at http://127.0.0.1:35047/ using document root <none> and password file <none>
I20260811 11:14:41.871657 22050 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:41.871726 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:41.871883 22050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:41.872984 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "c7a40f36527c4ddebdc7ae640e798b7c"
format_stamp: "Formatted at 2026-08-11 11:14:41 on dist-test-slave-69rg"
server_key: "ea74ea5b87b0eef32816e4d61b3db542"
server_key_iv: "49f8bbed60f9e0c8f678d90222b0e6a7"
server_key_version: "encryptionkey@0"
I20260811 11:14:41.873364 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "c7a40f36527c4ddebdc7ae640e798b7c"
format_stamp: "Formatted at 2026-08-11 11:14:41 on dist-test-slave-69rg"
server_key: "ea74ea5b87b0eef32816e4d61b3db542"
server_key_iv: "49f8bbed60f9e0c8f678d90222b0e6a7"
server_key_version: "encryptionkey@0"
I20260811 11:14:41.874826 22050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:41.875613 22069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:41.875813 22050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 11:14:41.875927 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "c7a40f36527c4ddebdc7ae640e798b7c"
format_stamp: "Formatted at 2026-08-11 11:14:41 on dist-test-slave-69rg"
server_key: "ea74ea5b87b0eef32816e4d61b3db542"
server_key_iv: "49f8bbed60f9e0c8f678d90222b0e6a7"
server_key_version: "encryptionkey@0"
I20260811 11:14:41.876003 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:41.886404 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:41.888752 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:41.888926 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:41.893250 22050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46751
I20260811 11:14:41.893281 22130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46751 every 8 connection(s)
I20260811 11:14:41.893641 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:41.894517 22131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:41.897111 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c: Bootstrap starting.
I20260811 11:14:41.897744 22131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:41.898092 22131 log.cc:824] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:41.898844 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c: No bootstrap required, opened a new log
I20260811 11:14:41.899406 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22050
I20260811 11:14:41.899541 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:41.899730 30276 external_mini_cluster.cc:1428] Setting key c05ec071ad9ac4d9023ccefc31179f68
I20260811 11:14:41.901085 22131 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } }
I20260811 11:14:41.901227 22131 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:41.901250 22131 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a40f36527c4ddebdc7ae640e798b7c, State: Initialized, Role: FOLLOWER
I20260811 11:14:41.901372 22131 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } }
I20260811 11:14:41.901455 22131 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:41.901479 22131 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:41.901530 22131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:41.902730 22131 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } }
I20260811 11:14:41.902854 22131 leader_election.cc:302] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7a40f36527c4ddebdc7ae640e798b7c; no voters:
I20260811 11:14:41.903008 22131 leader_election.cc:288] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:41.903080 22136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:41.903236 22131 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:41.903239 22136 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [term 1 LEADER]: Becoming Leader. State: Replica: c7a40f36527c4ddebdc7ae640e798b7c, State: Running, Role: LEADER
I20260811 11:14:41.903483 22136 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } }
I20260811 11:14:41.903860 22136 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7a40f36527c4ddebdc7ae640e798b7c. Latest consensus state: current_term: 1 leader_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } } }
I20260811 11:14:41.903937 22136 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:41.904222 22151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:41.904820 22151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:41.905689 22139 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a40f36527c4ddebdc7ae640e798b7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46751 } } }
I20260811 11:14:41.905833 22139 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:41.906940 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:41.900278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:57233 (local address 127.0.0.1:46751)
0811 11:14:41.900474 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:41.900482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:41.900508 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:14:41.901154 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:41.901163 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:41.901258 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:41.901441 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:41.901906 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:41.903385 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:41.904551 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:41.905125 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:41.905263 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:41.906163 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:41.906167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:41.906168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:41.906428 (+ 260us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:41.906471 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:41.906487 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:41.906583 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:41.906699 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:14:41.906728 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260811 11:14:41.906949 22151 catalog_manager.cc:1347] Generated new cluster ID: 75be2aaefd104424bc2cca51b4eabe73
I20260811 11:14:41.907228 22151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:41.907975 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:41.913583 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:41.909546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:50847 (local address 127.0.0.1:46751)
0811 11:14:41.909584 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:41.909586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:41.909661 (+ 75us) server_negotiation.cc:407] Connection header received
0811 11:14:41.909755 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:41.909756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:41.909767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:41.909796 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:41.910037 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:41.910751 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:41.911787 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:41.912431 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:41.912478 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:41.913286 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:41.913287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:41.913288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:41.913292 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:41.913319 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:41.913323 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:41.913358 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:41.913497 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 11:14:41.913511 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:14:41.925428 22151 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:41.926172 22151 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:41.929719 22151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7a40f36527c4ddebdc7ae640e798b7c: Generated new TSK 0
I20260811 11:14:41.929977 22151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:41.936170 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22050
W20260811 11:14:41.944557 22032 connection.cc:535] client connection to 127.0.0.1:46751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46751): Connection reset by peer (error 104)
2026-08-11T11:14:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:42 dist-test-slave-69rg krb5kdc[22170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:42 dist-test-slave-69rg krb5kdc[22170](info): set up 2 sockets
Aug 11 11:14:42 dist-test-slave-69rg krb5kdc[22170](info): commencing operation
krb5kdc: starting...
W20260811 11:14:43.056948 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:43 dist-test-slave-69rg krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:43.189075 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:43.273667 22183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:43.273972 22183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:43.274016 22183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:43.275328 22183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:43.275399 22183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:43.275424 22183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:43.275446 22183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:43.275477 22183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:43.277258 22183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:43.277558 22183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:43.277838 22183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:43.282950 22190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:43.283069 22183 server_base.cc:1034] running on GCE node
W20260811 11:14:43.282948 22189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:43.283185 22193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:43.283437 22183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:43.283747 22183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:43.284962 22183 hybrid_clock.cc:648] HybridClock initialized: now 1786446883284942 us; error 27 us; skew 500 ppm
Aug 11 11:14:43 dist-test-slave-69rg krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:43.287684 22183 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:43.288203 22183 webserver.cc:460] Webserver started at http://127.0.0.1:45149/ using document root <none> and password file <none>
I20260811 11:14:43.288494 22183 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:43.288566 22183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:43.288715 22183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:43.289770 22183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "0470ce8d00374474b029a45468a46af0"
format_stamp: "Formatted at 2026-08-11 11:14:43 on dist-test-slave-69rg"
server_key: "559d817174a619ed423a39ebfb8809f4"
server_key_iv: "fc30c69526e0bda08856258248d5445b"
server_key_version: "encryptionkey@0"
I20260811 11:14:43.290189 22183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "0470ce8d00374474b029a45468a46af0"
format_stamp: "Formatted at 2026-08-11 11:14:43 on dist-test-slave-69rg"
server_key: "559d817174a619ed423a39ebfb8809f4"
server_key_iv: "fc30c69526e0bda08856258248d5445b"
server_key_version: "encryptionkey@0"
I20260811 11:14:43.291606 22183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:43.292426 22201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:43.292598 22183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:43.292708 22183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "0470ce8d00374474b029a45468a46af0"
format_stamp: "Formatted at 2026-08-11 11:14:43 on dist-test-slave-69rg"
server_key: "559d817174a619ed423a39ebfb8809f4"
server_key_iv: "fc30c69526e0bda08856258248d5445b"
server_key_version: "encryptionkey@0"
I20260811 11:14:43.292786 22183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:43.296289 22183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:43.298748 22183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:43.298913 22183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:43.303448 22183 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38401
I20260811 11:14:43.303488 22261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38401 every 8 connection(s)
I20260811 11:14:43.303867 22183 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:43.304538 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22183
I20260811 11:14:43.304665 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:43.304868 30276 external_mini_cluster.cc:1428] Setting key 7fb7ab5b5e8c33c7681013c1d1a223de
I20260811 11:14:43.305050 22262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:43.307672 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:43.305432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:60819 (local address 127.0.0.1:38401)
0811 11:14:43.305639 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:43.305649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:43.305684 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:14:43.306291 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:43.306348 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:43.306386 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:43.306554 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:43.307189 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:43.307193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:43.307194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:43.307198 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:43.307247 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:43.307263 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:43.307350 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:43.307389 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 11:14:43.307432 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260811 11:14:43.308692 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:43.308851 22262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0: Bootstrap starting.
I20260811 11:14:43.309657 22262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:43.310070 22262 log.cc:824] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:43.310971 22262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0: No bootstrap required, opened a new log
I20260811 11:14:43.311494 22266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:43.310429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:54033 (local address 127.0.0.1:38401)
0811 11:14:43.310483 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:43.310485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:43.310607 (+ 122us) server_negotiation.cc:407] Connection header received
0811 11:14:43.310668 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:43.310669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:43.310674 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:43.310744 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:43.311294 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:43.311296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:43.311296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:43.311300 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:43.311315 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:43.311319 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:43.311385 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:43.311420 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 11:14:43.311433 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 11:14:43.314021 22262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } }
I20260811 11:14:43.314173 22262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:43.314199 22262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0470ce8d00374474b029a45468a46af0, State: Initialized, Role: FOLLOWER
I20260811 11:14:43.314339 22262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } }
I20260811 11:14:43.314419 22262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:43.314442 22262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:43.314507 22262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:43.315299 22262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } }
I20260811 11:14:43.315423 22262 leader_election.cc:302] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0470ce8d00374474b029a45468a46af0; no voters:
I20260811 11:14:43.315620 22262 leader_election.cc:288] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:43.315689 22273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:43.315888 22262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:43.315891 22273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [term 1 LEADER]: Becoming Leader. State: Replica: 0470ce8d00374474b029a45468a46af0, State: Running, Role: LEADER
I20260811 11:14:43.316392 22273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } }
I20260811 11:14:43.317075 22274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0470ce8d00374474b029a45468a46af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } } }
I20260811 11:14:43.317174 22274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:43.317359 22275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0470ce8d00374474b029a45468a46af0. Latest consensus state: current_term: 1 leader_uuid: "0470ce8d00374474b029a45468a46af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0470ce8d00374474b029a45468a46af0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38401 } } }
I20260811 11:14:43.317505 22275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:43.317551 22287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:43.318205 22287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:43.319696 22287 catalog_manager.cc:1347] Generated new cluster ID: dfe5c111e0874c5f8d52c23eb14bce1c
I20260811 11:14:43.319752 22287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:43.332399 22287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:43.332612 22287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:43.346944 22287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0470ce8d00374474b029a45468a46af0: Generated new TSK 0
I20260811 11:14:43.347184 22287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:43.349351 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22183
2026-08-11T11:14:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:43 dist-test-slave-69rg krb5kdc[22298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:43 dist-test-slave-69rg krb5kdc[22298](info): set up 2 sockets
Aug 11 11:14:43 dist-test-slave-69rg krb5kdc[22298](info): commencing operation
krb5kdc: starting...
W20260811 11:14:44.493213 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:44.624413 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:44.719233 22312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:44.719542 22312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:44.719580 22312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:44.720762 22312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:44.720829 22312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:44.720852 22312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:44.720875 22312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:44.720904 22312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:44.722520 22312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:44.722824 22312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:44.723086 22312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:44.728695 22318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:44.728864 22312 server_base.cc:1034] running on GCE node
W20260811 11:14:44.728677 22317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:44.728679 22320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:44.729218 22312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:44.729637 22312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:44.730823 22312 hybrid_clock.cc:648] HybridClock initialized: now 1786446884730796 us; error 42 us; skew 500 ppm
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:44.733860 22312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:44.734453 22312 webserver.cc:460] Webserver started at http://127.0.0.1:33511/ using document root <none> and password file <none>
I20260811 11:14:44.734707 22312 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:44.734776 22312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:44.734944 22312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:44.736176 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "8b5117bd4d2f469f8afe6a75b19e16e5"
format_stamp: "Formatted at 2026-08-11 11:14:44 on dist-test-slave-69rg"
server_key: "35d86279cc4b03ffb31d2c61a9878d2f"
server_key_iv: "04114a57fafb85217ddf07d6c0c566f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:44.736590 22312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "8b5117bd4d2f469f8afe6a75b19e16e5"
format_stamp: "Formatted at 2026-08-11 11:14:44 on dist-test-slave-69rg"
server_key: "35d86279cc4b03ffb31d2c61a9878d2f"
server_key_iv: "04114a57fafb85217ddf07d6c0c566f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:44.738144 22312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:44.739017 22328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:44.739214 22312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:44.739327 22312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "8b5117bd4d2f469f8afe6a75b19e16e5"
format_stamp: "Formatted at 2026-08-11 11:14:44 on dist-test-slave-69rg"
server_key: "35d86279cc4b03ffb31d2c61a9878d2f"
server_key_iv: "04114a57fafb85217ddf07d6c0c566f5"
server_key_version: "encryptionkey@0"
I20260811 11:14:44.739401 22312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:44.757784 22312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:44.760365 22312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:44.760568 22312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:44.766017 22312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33265
I20260811 11:14:44.766063 22392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33265 every 8 connection(s)
I20260811 11:14:44.766489 22312 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:44.767479 22393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:44.770292 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22312
I20260811 11:14:44.770458 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:44.770671 30276 external_mini_cluster.cc:1428] Setting key 1ff24853e66129d59937064b83ada705
I20260811 11:14:44.770661 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5: Bootstrap starting.
I20260811 11:14:44.771502 22393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:44.771839 22393 log.cc:824] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:44.772608 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5: No bootstrap required, opened a new log
I20260811 11:14:44.774703 22393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } }
I20260811 11:14:44.774876 22393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:44.774948 22393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5117bd4d2f469f8afe6a75b19e16e5, State: Initialized, Role: FOLLOWER
I20260811 11:14:44.775199 22393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } }
I20260811 11:14:44.775336 22393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:44.775388 22393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:44.775440 22393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:44.776144 22393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } }
I20260811 11:14:44.776269 22393 leader_election.cc:302] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b5117bd4d2f469f8afe6a75b19e16e5; no voters:
I20260811 11:14:44.776490 22393 leader_election.cc:288] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:44.776505 22399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:44.776677 22399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8b5117bd4d2f469f8afe6a75b19e16e5, State: Running, Role: LEADER
I20260811 11:14:44.776847 22393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:44.776831 22399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } }
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:14:44.779023 22401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } } }
I20260811 11:14:44.779150 22401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:44.779026 22400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b5117bd4d2f469f8afe6a75b19e16e5. Latest consensus state: current_term: 1 leader_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b5117bd4d2f469f8afe6a75b19e16e5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33265 } } }
I20260811 11:14:44.779316 22400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:44.779495 22414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:44.780103 22414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:44.782128 22414 catalog_manager.cc:1347] Generated new cluster ID: a5e49ee04f3a464a936e0f40a6374533
I20260811 11:14:44.782186 22414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:44.784368 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:44.771297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:60941 (local address 127.0.0.1:33265)
0811 11:14:44.771530 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:44.771541 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:44.771577 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:14:44.772194 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:44.772202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:44.772260 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:44.772422 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:44.773180 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:44.774225 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:44.775038 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:44.775617 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:44.775738 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:44.779095 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:44.779100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:44.779101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:44.779115 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:44.781818 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:44.782354 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:44.782358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:44.782358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:44.782473 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:44.782740 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:44.782741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:44.782742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:44.783093 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:44.783267 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:44.783882 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 11:14:44.784007 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260811 11:14:44.785508 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:44.794896 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:44.787272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:52795 (local address 127.0.0.1:33265)
0811 11:14:44.787311 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:44.787313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:44.787543 (+ 230us) server_negotiation.cc:407] Connection header received
0811 11:14:44.787591 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:44.787592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:44.787603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:44.787666 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:44.788286 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:44.789108 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:44.790130 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:44.790478 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:44.790557 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:44.791971 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:44.791972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:44.791973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:44.791984 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:44.793597 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:44.794014 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:44.794015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:44.794016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:44.794077 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:44.794301 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:44.794302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:44.794302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:44.794435 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:44.794556 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:44.794718 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:14:44.794787 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:14:44.799589 22414 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:44.800160 22414 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:44.803064 22414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b5117bd4d2f469f8afe6a75b19e16e5: Generated new TSK 0
I20260811 11:14:44.803288 22414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:44.805938 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22312
W20260811 11:14:44.817981 22296 connection.cc:535] client connection to 127.0.0.1:33265 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33265): Connection reset by peer (error 104)
2026-08-11T11:14:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22428](info): set up 2 sockets
Aug 11 11:14:44 dist-test-slave-69rg krb5kdc[22428](info): commencing operation
krb5kdc: starting...
W20260811 11:14:46.011442 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:46 dist-test-slave-69rg krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:46.164917 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:46.255308 22443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:46.255546 22443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:46.255584 22443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:46.256690 22443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:46.256758 22443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:46.256783 22443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:46.256811 22443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:46.256841 22443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:46.258427 22443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:46.258714 22443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:46.258953 22443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:46.264039 22449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:46.264065 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:46.264070 22454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:46.264104 22443 server_base.cc:1034] running on GCE node
I20260811 11:14:46.264518 22443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:46.264818 22443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:46.265976 22443 hybrid_clock.cc:648] HybridClock initialized: now 1786446886265965 us; error 30 us; skew 500 ppm
Aug 11 11:14:46 dist-test-slave-69rg krb5kdc[22428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:46.274411 22443 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:46.274895 22443 webserver.cc:460] Webserver started at http://127.0.0.1:33997/ using document root <none> and password file <none>
I20260811 11:14:46.275139 22443 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:46.275218 22443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:46.275365 22443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:46.276470 22443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "6c90ed6a4b3c40da8f2df2360d947ab9"
format_stamp: "Formatted at 2026-08-11 11:14:46 on dist-test-slave-69rg"
server_key: "1b2cc2ab3cd64fa527f662c7ac53872b"
server_key_iv: "fbb5a77b8c062f4a8feaa2d90d2ff30e"
server_key_version: "encryptionkey@0"
I20260811 11:14:46.276943 22443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "6c90ed6a4b3c40da8f2df2360d947ab9"
format_stamp: "Formatted at 2026-08-11 11:14:46 on dist-test-slave-69rg"
server_key: "1b2cc2ab3cd64fa527f662c7ac53872b"
server_key_iv: "fbb5a77b8c062f4a8feaa2d90d2ff30e"
server_key_version: "encryptionkey@0"
I20260811 11:14:46.278477 22443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:46.279277 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:46.279481 22443 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:14:46.279598 22443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "6c90ed6a4b3c40da8f2df2360d947ab9"
format_stamp: "Formatted at 2026-08-11 11:14:46 on dist-test-slave-69rg"
server_key: "1b2cc2ab3cd64fa527f662c7ac53872b"
server_key_iv: "fbb5a77b8c062f4a8feaa2d90d2ff30e"
server_key_version: "encryptionkey@0"
I20260811 11:14:46.279685 22443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:46.289876 22443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:46.291867 22443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:46.292026 22443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:46.296240 22443 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41949
I20260811 11:14:46.296277 22519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41949 every 8 connection(s)
I20260811 11:14:46.296648 22443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:46.297732 22520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:46.300119 22520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9: Bootstrap starting.
I20260811 11:14:46.300740 22520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:46.300748 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22443
I20260811 11:14:46.300864 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:46.301059 30276 external_mini_cluster.cc:1428] Setting key 3106e88116fc658f0ddc48ed8679ad01
I20260811 11:14:46.301101 22520 log.cc:824] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:46.302089 22520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9: No bootstrap required, opened a new log
I20260811 11:14:46.304788 22520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } }
I20260811 11:14:46.304926 22520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:46.304948 22520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c90ed6a4b3c40da8f2df2360d947ab9, State: Initialized, Role: FOLLOWER
I20260811 11:14:46.305076 22520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } }
I20260811 11:14:46.305151 22520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:46.305186 22520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:46.305238 22520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:46.305951 22520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } }
I20260811 11:14:46.306066 22520 leader_election.cc:302] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c90ed6a4b3c40da8f2df2360d947ab9; no voters:
I20260811 11:14:46.306244 22520 leader_election.cc:288] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:46.306310 22526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:46.306550 22526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [term 1 LEADER]: Becoming Leader. State: Replica: 6c90ed6a4b3c40da8f2df2360d947ab9, State: Running, Role: LEADER
I20260811 11:14:46.306689 22520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:46.307006 22526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } }
I20260811 11:14:46.307403 22526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c90ed6a4b3c40da8f2df2360d947ab9. Latest consensus state: current_term: 1 leader_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } } }
I20260811 11:14:46.307731 22528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90ed6a4b3c40da8f2df2360d947ab9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41949 } } }
I20260811 11:14:46.307866 22526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:46.308032 22528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:46.308144 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:46.301661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:59031 (local address 127.0.0.1:41949)
0811 11:14:46.301874 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:46.301884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:46.301918 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:14:46.302546 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:46.302555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:46.302621 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:46.302779 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:46.302979 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:46.304427 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:46.305509 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:46.306234 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:46.306347 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:46.307072 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:46.307077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:46.307077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:46.307301 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:46.307340 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:46.307354 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:46.307445 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:46.307835 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 11:14:46.307873 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260811 11:14:46.308640 22537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:46.309160 22537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:46.309401 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:46.311213 22537 catalog_manager.cc:1347] Generated new cluster ID: b74c32b3605549d6b19dd9b47d226418
I20260811 11:14:46.311266 22537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:46.314888 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:46.311131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:35821 (local address 127.0.0.1:41949)
0811 11:14:46.311167 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:46.311169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:46.311425 (+ 256us) server_negotiation.cc:407] Connection header received
0811 11:14:46.311490 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:46.311491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:46.311502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:46.311563 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:46.311709 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:46.312446 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:46.313487 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:46.313807 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:46.313890 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:46.314633 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:46.314634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:46.314634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:46.314639 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:46.314655 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:46.314659 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:46.314713 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:46.314808 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 11:14:46.314818 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:14:46.319058 22537 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:46.319762 22537 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:46.326723 22537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c90ed6a4b3c40da8f2df2360d947ab9: Generated new TSK 0
I20260811 11:14:46.326901 22537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:46.333477 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22443
W20260811 11:14:46.341418 22426 connection.cc:535] client connection to 127.0.0.1:41949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41949): Connection reset by peer (error 104)
2026-08-11T11:14:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:46 dist-test-slave-69rg krb5kdc[22558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:46 dist-test-slave-69rg krb5kdc[22558](info): set up 2 sockets
Aug 11 11:14:46 dist-test-slave-69rg krb5kdc[22558](info): commencing operation
krb5kdc: starting...
W20260811 11:14:47.494124 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:47 dist-test-slave-69rg krb5kdc[22558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:14:47.633499 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:47.728669 22571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:47.728905 22571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:47.728931 22571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:47.730058 22571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:47.730093 22571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:47.730108 22571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:47.730120 22571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:47.730132 22571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:47.731560 22571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:47.731768 22571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:47.731956 22571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:47.736965 22581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:47.736999 22579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:47.736974 22577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:47.737391 22571 server_base.cc:1034] running on GCE node
I20260811 11:14:47.737633 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:47.737977 22571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:47.739132 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786446887739116 us; error 31 us; skew 500 ppm
Aug 11 11:14:47 dist-test-slave-69rg krb5kdc[22558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:47.741892 22571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:14:47.742370 22571 webserver.cc:460] Webserver started at http://127.0.0.1:40145/ using document root <none> and password file <none>
I20260811 11:14:47.742609 22571 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:47.742676 22571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:47.742841 22571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:47.743976 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab"
format_stamp: "Formatted at 2026-08-11 11:14:47 on dist-test-slave-69rg"
server_key: "9e6cfae471c0614846de1175f73606df"
server_key_iv: "19529c5ccad17a8897fae3f5249cef89"
server_key_version: "encryptionkey@0"
I20260811 11:14:47.744364 22571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab"
format_stamp: "Formatted at 2026-08-11 11:14:47 on dist-test-slave-69rg"
server_key: "9e6cfae471c0614846de1175f73606df"
server_key_iv: "19529c5ccad17a8897fae3f5249cef89"
server_key_version: "encryptionkey@0"
I20260811 11:14:47.745877 22571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:47.746693 22590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:47.746871 22571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 11:14:47.746978 22571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab"
format_stamp: "Formatted at 2026-08-11 11:14:47 on dist-test-slave-69rg"
server_key: "9e6cfae471c0614846de1175f73606df"
server_key_iv: "19529c5ccad17a8897fae3f5249cef89"
server_key_version: "encryptionkey@0"
I20260811 11:14:47.747061 22571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:47.750743 22571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:47.752774 22571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:47.752934 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:47.757361 22571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38467
I20260811 11:14:47.757478 22651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38467 every 8 connection(s)
I20260811 11:14:47.757809 22571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:47.758999 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:47.759897 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22571
I20260811 11:14:47.760051 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:47.760373 30276 external_mini_cluster.cc:1428] Setting key b446d0ce5bea4b626cf43b5fdd1c2cf5
I20260811 11:14:47.762017 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab: Bootstrap starting.
I20260811 11:14:47.762903 22652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:47.763087 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:47.760974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:54221 (local address 127.0.0.1:38467)
0811 11:14:47.761152 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:47.761160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:47.761199 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:14:47.761676 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:47.761683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:47.761708 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:47.761891 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:47.762563 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:47.762568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:47.762569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:47.762574 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:47.762626 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:47.762642 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:47.762742 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:47.762810 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 11:14:47.762862 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260811 11:14:47.763266 22652 log.cc:824] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:47.763994 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab: No bootstrap required, opened a new log
I20260811 11:14:47.764098 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:47.766700 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:47.765663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.162:50825 (local address 127.0.0.1:38467)
0811 11:14:47.765702 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:47.765704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:47.765807 (+ 103us) server_negotiation.cc:407] Connection header received
0811 11:14:47.765888 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:47.765889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:47.765893 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:47.765946 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:47.766489 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:47.766491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:47.766491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:47.766494 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:47.766506 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:47.766510 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:47.766553 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:47.766603 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 11:14:47.766614 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:14:47.766857 22652 raft_consensus.cc:348] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 11:14:47.767036 22652 raft_consensus.cc:374] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:47.767113 22652 raft_consensus.cc:729] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d750bb4ff86b44a7971cdc9d34dcf7ab, State: Initialized, Role: FOLLOWER
I20260811 11:14:47.767271 22652 consensus_queue.cc:260] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 11:14:47.767360 22652 raft_consensus.cc:388] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:47.767400 22652 raft_consensus.cc:482] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:47.767449 22652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:47.768015 22652 raft_consensus.cc:504] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 11:14:47.768133 22652 leader_election.cc:302] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d750bb4ff86b44a7971cdc9d34dcf7ab; no voters:
I20260811 11:14:47.768324 22652 leader_election.cc:288] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:47.768349 22662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:47.768492 22662 raft_consensus.cc:686] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [term 1 LEADER]: Becoming Leader. State: Replica: d750bb4ff86b44a7971cdc9d34dcf7ab, State: Running, Role: LEADER
I20260811 11:14:47.768580 22652 sys_catalog.cc:564] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:47.768633 22662 consensus_queue.cc:237] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } }
I20260811 11:14:47.769086 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader d750bb4ff86b44a7971cdc9d34dcf7ab. Latest consensus state: current_term: 1 leader_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260811 11:14:47.769196 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:47.769402 22673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:47.769600 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d750bb4ff86b44a7971cdc9d34dcf7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38467 } } }
I20260811 11:14:47.769687 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:47.769982 22673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:47.771588 22673 catalog_manager.cc:1347] Generated new cluster ID: ac2b7adc9e6048349db67a06a790d54a
I20260811 11:14:47.771641 22673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:47.787010 22673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:47.787206 22673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:47.797176 22673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d750bb4ff86b44a7971cdc9d34dcf7ab: Generated new TSK 0
I20260811 11:14:47.797412 22673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:47.802618 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22571
2026-08-11T11:14:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:47 dist-test-slave-69rg krb5kdc[22690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:47 dist-test-slave-69rg krb5kdc[22690](info): set up 2 sockets
Aug 11 11:14:47 dist-test-slave-69rg krb5kdc[22690](info): commencing operation
krb5kdc: starting...
W20260811 11:14:48.930806 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:49.082173 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:46819
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:41557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:46819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:49.179586 22704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:49.179912 22704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:49.179955 22704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:49.181214 22704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:49.181295 22704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:49.181319 22704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:49.181342 22704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:49.181370 22704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:49.183100 22704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:41557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:46819
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:46819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:49.183374 22704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:49.183598 22704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:49.189235 22709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:49.189229 22710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:49.189229 22714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:49.189527 22704 server_base.cc:1034] running on GCE node
I20260811 11:14:49.189863 22704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:49.190246 22704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:49.191432 22704 hybrid_clock.cc:648] HybridClock initialized: now 1786446889191427 us; error 45 us; skew 500 ppm
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446889, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:49.194461 22704 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:49.195000 22704 webserver.cc:460] Webserver started at http://127.29.145.62:33111/ using document root <none> and password file <none>
I20260811 11:14:49.195231 22704 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:49.195276 22704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:49.195384 22704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:49.196655 22704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "ab6bb257a2014006a0309319244a863c"
format_stamp: "Formatted at 2026-08-11 11:14:49 on dist-test-slave-69rg"
server_key: "87ddcf20acc8941d5df9314fcb352fe7"
server_key_iv: "1761695ba2d0a9b6ef56c51ab4b10405"
server_key_version: "encryptionkey@0"
I20260811 11:14:49.197022 22704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "ab6bb257a2014006a0309319244a863c"
format_stamp: "Formatted at 2026-08-11 11:14:49 on dist-test-slave-69rg"
server_key: "87ddcf20acc8941d5df9314fcb352fe7"
server_key_iv: "1761695ba2d0a9b6ef56c51ab4b10405"
server_key_version: "encryptionkey@0"
I20260811 11:14:49.198658 22704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 11:14:49.199458 22723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:49.199641 22704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:49.199738 22704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "ab6bb257a2014006a0309319244a863c"
format_stamp: "Formatted at 2026-08-11 11:14:49 on dist-test-slave-69rg"
server_key: "87ddcf20acc8941d5df9314fcb352fe7"
server_key_iv: "1761695ba2d0a9b6ef56c51ab4b10405"
server_key_version: "encryptionkey@0"
I20260811 11:14:49.199805 22704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:49.213547 22704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:49.215878 22704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:49.216037 22704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:49.221124 22704 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:46819
I20260811 11:14:49.221222 22780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:46819 every 8 connection(s)
I20260811 11:14:49.221666 22704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:49.222838 22781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:49.225805 22781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c: Bootstrap starting.
I20260811 11:14:49.226650 22781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:49.227013 22781 log.cc:824] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:49.227927 22781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c: No bootstrap required, opened a new log
I20260811 11:14:49.230072 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22704
I20260811 11:14:49.230212 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:49.230057 22781 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } }
I20260811 11:14:49.230279 22781 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:49.230347 22781 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6bb257a2014006a0309319244a863c, State: Initialized, Role: FOLLOWER
I20260811 11:14:49.230455 30276 external_mini_cluster.cc:1428] Setting key adf7e50a86e2be3777d31b65e11f05cd
I20260811 11:14:49.230563 22781 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } }
I20260811 11:14:49.230666 22781 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:49.230718 22781 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:49.230775 22781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:49.231711 22781 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } }
I20260811 11:14:49.231848 22781 leader_election.cc:302] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab6bb257a2014006a0309319244a863c; no voters:
I20260811 11:14:49.232039 22781 leader_election.cc:288] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:49.232090 22786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:49.232239 22786 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [term 1 LEADER]: Becoming Leader. State: Replica: ab6bb257a2014006a0309319244a863c, State: Running, Role: LEADER
I20260811 11:14:49.232370 22781 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:49.232391 22786 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } }
I20260811 11:14:49.232894 22787 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab6bb257a2014006a0309319244a863c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } } }
I20260811 11:14:49.233009 22787 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:49.233255 22790 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab6bb257a2014006a0309319244a863c. Latest consensus state: current_term: 1 leader_uuid: "ab6bb257a2014006a0309319244a863c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab6bb257a2014006a0309319244a863c" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 46819 } } }
I20260811 11:14:49.233640 22790 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:49.234019 22796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:49.234689 22796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:49.236655 22796 catalog_manager.cc:1347] Generated new cluster ID: eaee04f74ee843658c085c28f86746d6
I20260811 11:14:49.236718 22796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:49.246049 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:49.231172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49689 (local address 127.29.145.62:46819)
0811 11:14:49.231911 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:49.231923 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:49.231966 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:14:49.232138 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:49.232147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:49.232223 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:49.233333 (+ 1110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:49.233462 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:49.235105 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:49.236397 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:49.237091 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:49.237204 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:49.240640 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:49.240648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:49.240649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:49.240668 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:49.243761 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:49.244300 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:49.244305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:49.244305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:49.244459 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:49.244713 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:49.244714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:49.244714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:49.245125 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:49.245352 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:49.245455 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:14:49.245727 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":607,"threads_started":1}
I20260811 11:14:49.247452 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:49.257445 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:49.249466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59423 (local address 127.29.145.62:46819)
0811 11:14:49.249514 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:49.249516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:49.249618 (+ 102us) server_negotiation.cc:407] Connection header received
0811 11:14:49.249668 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:49.249669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:49.249681 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:49.249747 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:49.250441 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:49.251297 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:49.252332 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:49.252702 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:49.252780 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:49.254382 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:49.254385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:49.254385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:49.254393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:49.256110 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:49.256539 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:49.256540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:49.256541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:49.256608 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:49.256844 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:49.256845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:49.256846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:49.257016 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:49.257147 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:49.257258 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 11:14:49.257329 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:14:49.265599 22796 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:49.266283 22796 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:49.273509 22796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab6bb257a2014006a0309319244a863c: Generated new TSK 0
I20260811 11:14:49.273793 22796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:49.279770 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22704
W20260811 11:14:49.290694 22686 connection.cc:535] client connection to 127.29.145.62:46819 recv error: Network error: failed to read from TLS socket (remote: 127.29.145.62:46819): Connection reset by peer (error 104)
2026-08-11T11:14:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22817](info): set up 2 sockets
Aug 11 11:14:49 dist-test-slave-69rg krb5kdc[22817](info): commencing operation
krb5kdc: starting...
W20260811 11:14:50.443939 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:50 dist-test-slave-69rg krb5kdc[22817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:50.577232 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42077
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:40289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:42077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:50.665980 22833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:50.666277 22833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:50.666323 22833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:50.667488 22833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:50.667557 22833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:50.667583 22833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:50.667613 22833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:50.667634 22833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:50.669121 22833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:40289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:42077
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:42077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:50.669417 22833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:50.669648 22833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:14:50.674924 22833 server_base.cc:1034] running on GCE node
W20260811 11:14:50.674901 22839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:50.674882 22840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:50.674890 22842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:50.675441 22833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:50.675756 22833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:50.676934 22833 hybrid_clock.cc:648] HybridClock initialized: now 1786446890676916 us; error 38 us; skew 500 ppm
Aug 11 11:14:50 dist-test-slave-69rg krb5kdc[22817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446890, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:50.679651 22833 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:50.680099 22833 webserver.cc:460] Webserver started at http://127.29.145.62:40071/ using document root <none> and password file <none>
I20260811 11:14:50.680361 22833 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:50.680431 22833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:50.680578 22833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:50.681689 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "963cc0d892e542b7a61c56ee5f6dbac3"
format_stamp: "Formatted at 2026-08-11 11:14:50 on dist-test-slave-69rg"
server_key: "ea396c58524953c92fdb3d02f891c49a"
server_key_iv: "bd5c4e0171307896bfd12f613f061285"
server_key_version: "encryptionkey@0"
I20260811 11:14:50.682120 22833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "963cc0d892e542b7a61c56ee5f6dbac3"
format_stamp: "Formatted at 2026-08-11 11:14:50 on dist-test-slave-69rg"
server_key: "ea396c58524953c92fdb3d02f891c49a"
server_key_iv: "bd5c4e0171307896bfd12f613f061285"
server_key_version: "encryptionkey@0"
I20260811 11:14:50.683524 22833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:50.684350 22850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:50.684535 22833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:50.684641 22833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "963cc0d892e542b7a61c56ee5f6dbac3"
format_stamp: "Formatted at 2026-08-11 11:14:50 on dist-test-slave-69rg"
server_key: "ea396c58524953c92fdb3d02f891c49a"
server_key_iv: "bd5c4e0171307896bfd12f613f061285"
server_key_version: "encryptionkey@0"
I20260811 11:14:50.684715 22833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:50.716647 22833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:50.719091 22833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:50.719280 22833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:50.723482 22833 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:42077
I20260811 11:14:50.723536 22912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:42077 every 8 connection(s)
I20260811 11:14:50.723933 22833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:50.724890 22913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:50.727326 22913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3: Bootstrap starting.
I20260811 11:14:50.727871 22913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:50.728128 22913 log.cc:824] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:50.728847 22913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3: No bootstrap required, opened a new log
I20260811 11:14:50.730507 22913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } }
I20260811 11:14:50.730651 22913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:50.730675 22913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 963cc0d892e542b7a61c56ee5f6dbac3, State: Initialized, Role: FOLLOWER
I20260811 11:14:50.730813 22913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } }
I20260811 11:14:50.730897 22913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:50.730921 22913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:50.730950 22913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:50.731768 22913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } }
I20260811 11:14:50.731860 22913 leader_election.cc:302] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 963cc0d892e542b7a61c56ee5f6dbac3; no voters:
I20260811 11:14:50.732016 22913 leader_election.cc:288] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:50.732041 22916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:50.732160 22916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [term 1 LEADER]: Becoming Leader. State: Replica: 963cc0d892e542b7a61c56ee5f6dbac3, State: Running, Role: LEADER
I20260811 11:14:50.732249 22916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } }
I20260811 11:14:50.732306 22913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:50.732788 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } } }
I20260811 11:14:50.732817 22918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 963cc0d892e542b7a61c56ee5f6dbac3. Latest consensus state: current_term: 1 leader_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963cc0d892e542b7a61c56ee5f6dbac3" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 42077 } } }
I20260811 11:14:50.732992 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:50.733006 22918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:50.733397 22924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:50.733464 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22833
I20260811 11:14:50.733600 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:50.733901 30276 external_mini_cluster.cc:1428] Setting key c0134672786379e305f11728d2bbeeb0
I20260811 11:14:50.734203 22924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:50.736008 22924 catalog_manager.cc:1347] Generated new cluster ID: c357df7d8fab49e3b34943517363ecd5
I20260811 11:14:50.736069 22924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:50.750718 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:50.734581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53703 (local address 127.29.145.62:42077)
0811 11:14:50.735072 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:50.735082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:50.735119 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:14:50.735257 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:50.735265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:50.735340 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:50.735501 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:50.735750 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:50.746816 (+ 11066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:50.748164 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:50.748868 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:50.749005 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:50.749877 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:50.749883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:50.749884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:50.750140 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:50.750183 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:50.750200 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:50.750286 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:50.750412 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 11:14:50.750447 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":102,"threads_started":1}
I20260811 11:14:50.751744 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:50.757678 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:50.753469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47291 (local address 127.29.145.62:42077)
0811 11:14:50.753505 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:50.753507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:50.753622 (+ 115us) server_negotiation.cc:407] Connection header received
0811 11:14:50.753677 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:50.753678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:50.753690 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:50.753762 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:50.753913 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:50.754689 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:50.755734 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:50.756538 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:50.756613 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:50.757375 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:50.757377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:50.757377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:50.757382 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:50.757399 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:50.757403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:50.757468 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:50.757552 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 11:14:50.757566 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 11:14:50.759438 22924 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:50.760181 22924 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:50.766566 22924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 963cc0d892e542b7a61c56ee5f6dbac3: Generated new TSK 0
I20260811 11:14:50.766786 22924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:50.776320 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22833
W20260811 11:14:50.786152 22813 connection.cc:535] client connection to 127.29.145.62:42077 recv error: Network error: failed to read from TLS socket (remote: 127.29.145.62:42077): Connection reset by peer (error 104)
2026-08-11T11:14:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:50 dist-test-slave-69rg krb5kdc[22946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:50 dist-test-slave-69rg krb5kdc[22946](info): set up 2 sockets
Aug 11 11:14:50 dist-test-slave-69rg krb5kdc[22946](info): commencing operation
krb5kdc: starting...
W20260811 11:14:51.912467 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:52 dist-test-slave-69rg krb5kdc[22946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:52.042021 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:35529
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:45449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:35529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:52.133180 22959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:52.133432 22959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:52.133459 22959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:52.134655 22959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:52.134801 22959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:52.134861 22959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:52.134883 22959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:52.134896 22959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:52.137084 22959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:45449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:35529
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:35529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:52.137411 22959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:52.137672 22959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:52.142891 22966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:52.142892 22969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:52.142915 22965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:52.143743 22959 server_base.cc:1034] running on GCE node
I20260811 11:14:52.144053 22959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:52.144384 22959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:52.145573 22959 hybrid_clock.cc:648] HybridClock initialized: now 1786446892145546 us; error 70 us; skew 500 ppm
Aug 11 11:14:52 dist-test-slave-69rg krb5kdc[22946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446892, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:52.148500 22959 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:52.148962 22959 webserver.cc:460] Webserver started at http://127.29.145.62:38053/ using document root <none> and password file <none>
I20260811 11:14:52.149192 22959 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:52.149259 22959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:52.149412 22959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:52.150612 22959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "d0ad8605bd8a49108f117639281da80a"
format_stamp: "Formatted at 2026-08-11 11:14:52 on dist-test-slave-69rg"
server_key: "a42214522226acf470f8996878bf8e0b"
server_key_iv: "c610f794c3cf9b0fb79104e47916b572"
server_key_version: "encryptionkey@0"
I20260811 11:14:52.150990 22959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "d0ad8605bd8a49108f117639281da80a"
format_stamp: "Formatted at 2026-08-11 11:14:52 on dist-test-slave-69rg"
server_key: "a42214522226acf470f8996878bf8e0b"
server_key_iv: "c610f794c3cf9b0fb79104e47916b572"
server_key_version: "encryptionkey@0"
I20260811 11:14:52.152611 22959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:14:52.153959 22977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:52.154225 22959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:52.154407 22959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "d0ad8605bd8a49108f117639281da80a"
format_stamp: "Formatted at 2026-08-11 11:14:52 on dist-test-slave-69rg"
server_key: "a42214522226acf470f8996878bf8e0b"
server_key_iv: "c610f794c3cf9b0fb79104e47916b572"
server_key_version: "encryptionkey@0"
I20260811 11:14:52.154505 22959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:52.158324 22959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:52.160593 22959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:52.160755 22959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:52.165527 22959 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:35529
I20260811 11:14:52.165572 23038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:35529 every 8 connection(s)
I20260811 11:14:52.165953 22959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:52.167073 23039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:52.167809 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 22959
I20260811 11:14:52.168042 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:52.168259 30276 external_mini_cluster.cc:1428] Setting key 8e083e78080c86de5ad2b3425295a421
I20260811 11:14:52.170555 23039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a: Bootstrap starting.
I20260811 11:14:52.171113 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:52.168820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54185 (local address 127.29.145.62:35529)
0811 11:14:52.169034 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:52.169044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:52.169077 (+ 33us) server_negotiation.cc:407] Connection header received
0811 11:14:52.169625 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:52.169634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:52.169682 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:52.169869 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:52.170602 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:52.170606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:52.170607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:52.170611 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:52.170658 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:52.170671 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:52.170760 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:52.170854 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 11:14:52.170904 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260811 11:14:52.171358 23039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:52.171682 23039 log.cc:824] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:52.172098 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:52.172410 23039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a: No bootstrap required, opened a new log
I20260811 11:14:52.174804 23039 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } }
I20260811 11:14:52.175000 23039 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:52.175051 23039 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ad8605bd8a49108f117639281da80a, State: Initialized, Role: FOLLOWER
I20260811 11:14:52.175209 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:52.173647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54163 (local address 127.29.145.62:35529)
0811 11:14:52.173705 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:52.173707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:52.173799 (+ 92us) server_negotiation.cc:407] Connection header received
0811 11:14:52.173911 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:52.173912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:52.173918 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:52.173986 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:52.174823 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:52.174825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:52.174826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:52.174831 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:52.174847 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:52.174851 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:52.174946 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:52.175135 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 11:14:52.175150 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:14:52.175215 23039 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } }
I20260811 11:14:52.175355 23039 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:52.175405 23039 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:52.175452 23039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:52.176237 23039 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } }
I20260811 11:14:52.176374 23039 leader_election.cc:302] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0ad8605bd8a49108f117639281da80a; no voters:
I20260811 11:14:52.176563 23039 leader_election.cc:288] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:52.176609 23051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:52.176788 23051 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [term 1 LEADER]: Becoming Leader. State: Replica: d0ad8605bd8a49108f117639281da80a, State: Running, Role: LEADER
I20260811 11:14:52.176844 23039 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:52.176955 23051 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } }
I20260811 11:14:52.177446 23053 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0ad8605bd8a49108f117639281da80a. Latest consensus state: current_term: 1 leader_uuid: "d0ad8605bd8a49108f117639281da80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } } }
I20260811 11:14:52.177560 23053 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:52.177776 23052 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0ad8605bd8a49108f117639281da80a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ad8605bd8a49108f117639281da80a" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 35529 } } }
I20260811 11:14:52.177918 23052 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:52.178054 23061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:52.178604 23061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:52.180173 23061 catalog_manager.cc:1347] Generated new cluster ID: dc0ff8a4b1984333a8e65058eab02e01
I20260811 11:14:52.180289 23061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:52.183388 23061 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:52.183557 23061 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:52.188194 23061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0ad8605bd8a49108f117639281da80a: Generated new TSK 0
I20260811 11:14:52.188416 23061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:52.195309 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 22959
2026-08-11T11:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:52 dist-test-slave-69rg krb5kdc[23075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:52 dist-test-slave-69rg krb5kdc[23075](info): set up 2 sockets
Aug 11 11:14:52 dist-test-slave-69rg krb5kdc[23075](info): commencing operation
krb5kdc: starting...
W20260811 11:14:53.325163 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:53.457690 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:33833
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:38091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:33833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:53.541177 23090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:53.541433 23090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:53.541472 23090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:53.542644 23090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:53.542714 23090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:53.542742 23090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:53.542773 23090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:53.542802 23090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:53.544276 23090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:38091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:33833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:33833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:53.544567 23090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:53.544806 23090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:53.549774 23096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:53.549856 23095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:53.549837 23090 server_base.cc:1034] running on GCE node
W20260811 11:14:53.549785 23098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:53.550355 23090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:53.550644 23090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:53.551813 23090 hybrid_clock.cc:648] HybridClock initialized: now 1786446893551809 us; error 40 us; skew 500 ppm
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446893, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:53.554445 23090 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:53.554919 23090 webserver.cc:460] Webserver started at http://127.29.145.62:37753/ using document root <none> and password file <none>
I20260811 11:14:53.555140 23090 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:53.555208 23090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:53.555366 23090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:53.556457 23090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e"
format_stamp: "Formatted at 2026-08-11 11:14:53 on dist-test-slave-69rg"
server_key: "52b912be9cdae732cd18cd1f8b578012"
server_key_iv: "35a92d7b0c3585bed48b4de893c0882a"
server_key_version: "encryptionkey@0"
I20260811 11:14:53.556824 23090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e"
format_stamp: "Formatted at 2026-08-11 11:14:53 on dist-test-slave-69rg"
server_key: "52b912be9cdae732cd18cd1f8b578012"
server_key_iv: "35a92d7b0c3585bed48b4de893c0882a"
server_key_version: "encryptionkey@0"
I20260811 11:14:53.558231 23090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:53.559080 23105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:53.559263 23090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:53.559396 23090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e"
format_stamp: "Formatted at 2026-08-11 11:14:53 on dist-test-slave-69rg"
server_key: "52b912be9cdae732cd18cd1f8b578012"
server_key_iv: "35a92d7b0c3585bed48b4de893c0882a"
server_key_version: "encryptionkey@0"
I20260811 11:14:53.559475 23090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:53.568939 23090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:53.571313 23090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:53.571483 23090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:53.575852 23090 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:33833
I20260811 11:14:53.575889 23171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:33833 every 8 connection(s)
I20260811 11:14:53.576283 23090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:53.577212 23172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:53.579646 23172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: Bootstrap starting.
I20260811 11:14:53.580199 23172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:53.580466 23172 log.cc:824] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:53.581202 23172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: No bootstrap required, opened a new log
I20260811 11:14:53.582868 23172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } }
I20260811 11:14:53.583011 23172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:53.583034 23172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190ee8d39d6943a1bbd5ab760c9e3f6e, State: Initialized, Role: FOLLOWER
I20260811 11:14:53.583153 23172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } }
I20260811 11:14:53.583213 23172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:53.583235 23172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:53.583261 23172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:53.583527 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23090
I20260811 11:14:53.583645 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:53.583819 30276 external_mini_cluster.cc:1428] Setting key 78933894b6f0cd18e732e735a17daa38
I20260811 11:14:53.584507 23172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } }
I20260811 11:14:53.584640 23172 leader_election.cc:302] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 190ee8d39d6943a1bbd5ab760c9e3f6e; no voters:
I20260811 11:14:53.584833 23172 leader_election.cc:288] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:53.584856 23177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:53.584966 23177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [term 1 LEADER]: Becoming Leader. State: Replica: 190ee8d39d6943a1bbd5ab760c9e3f6e, State: Running, Role: LEADER
I20260811 11:14:53.585103 23172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:53.585119 23177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } }
I20260811 11:14:53.585461 23177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 190ee8d39d6943a1bbd5ab760c9e3f6e. Latest consensus state: current_term: 1 leader_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } } }
I20260811 11:14:53.585543 23177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:53.585472 23180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190ee8d39d6943a1bbd5ab760c9e3f6e" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 33833 } } }
I20260811 11:14:53.585610 23180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e [sys.catalog]: This master's current role is: LEADER
W20260811 11:14:53.587144 23198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:53.587214 23198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:53.587316 23183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:53.587482 23183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:53.589435 23183 catalog_manager.cc:1347] Generated new cluster ID: 473b9a1a3aae4d8b8e26186a92c6e8b1
I20260811 11:14:53.589495 23183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:53.596122 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:53.584358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33029 (local address 127.29.145.62:33833)
0811 11:14:53.584623 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:53.584634 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:53.584674 (+ 40us) server_negotiation.cc:407] Connection header received
0811 11:14:53.584807 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:53.584813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:53.584855 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:53.585003 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:53.585850 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:53.587448 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:53.588420 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:53.589018 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:53.589143 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:53.591925 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:53.591929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:53.591930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:53.591942 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:53.594410 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:53.594803 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:53.594807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:53.594807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:53.594934 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:53.595163 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:53.595164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:53.595164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:53.595433 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:53.595606 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:53.595722 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:14:53.595881 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":133,"threads_started":1}
I20260811 11:14:53.597084 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:53.607565 23183 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:53.608281 23183 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:53.613165 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:53.598598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52263 (local address 127.29.145.62:33833)
0811 11:14:53.598641 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:53.598644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:53.606063 (+ 7419us) server_negotiation.cc:407] Connection header received
0811 11:14:53.606162 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:53.606163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:53.606181 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:53.606253 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:53.606990 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:53.607673 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:53.608444 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:53.608637 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:53.608717 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:53.610407 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:53.610408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:53.610409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:53.610424 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:53.611934 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:53.612320 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:53.612322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:53.612322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:53.612386 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:53.612598 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:53.612599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:53.612599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:53.612723 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:53.612858 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:53.612955 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 11:14:53.613057 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:14:53.614866 23183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 190ee8d39d6943a1bbd5ab760c9e3f6e: Generated new TSK 0
I20260811 11:14:53.615082 23183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:53.618409 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23090
2026-08-11T11:14:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23209](info): set up 2 sockets
Aug 11 11:14:53 dist-test-slave-69rg krb5kdc[23209](info): commencing operation
krb5kdc: starting...
W20260811 11:14:54.730891 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:54 dist-test-slave-69rg krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:54.856583 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37341
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:37341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:54.939850 23224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:54.940050 23224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:54.940073 23224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:54.941149 23224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:54.941185 23224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:54.941200 23224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:54.941213 23224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:54.941226 23224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:54.942658 23224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:37341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:37341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:54.942849 23224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:54.943013 23224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:54.947850 23232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:54.947865 23230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:54.947906 23224 server_base.cc:1034] running on GCE node
W20260811 11:14:54.947868 23229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:54.948357 23224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:54.948645 23224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:54.949798 23224 hybrid_clock.cc:648] HybridClock initialized: now 1786446894949785 us; error 28 us; skew 500 ppm
Aug 11 11:14:54 dist-test-slave-69rg krb5kdc[23209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446894, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:54.952459 23224 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:54.952962 23224 webserver.cc:460] Webserver started at http://127.29.145.62:46833/ using document root <none> and password file <none>
I20260811 11:14:54.953193 23224 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:54.953261 23224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:54.953425 23224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:54.954775 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "6209e7eb25bc4ba18ea03c9ba6b33737"
format_stamp: "Formatted at 2026-08-11 11:14:54 on dist-test-slave-69rg"
server_key: "0449d63fc2459d7c469483cd9a3b32f9"
server_key_iv: "1f2175ebaf8d886d21f31a2588783504"
server_key_version: "encryptionkey@0"
I20260811 11:14:54.955168 23224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "6209e7eb25bc4ba18ea03c9ba6b33737"
format_stamp: "Formatted at 2026-08-11 11:14:54 on dist-test-slave-69rg"
server_key: "0449d63fc2459d7c469483cd9a3b32f9"
server_key_iv: "1f2175ebaf8d886d21f31a2588783504"
server_key_version: "encryptionkey@0"
I20260811 11:14:54.956673 23224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:14:54.957487 23239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:54.957769 23224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:54.957938 23224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "6209e7eb25bc4ba18ea03c9ba6b33737"
format_stamp: "Formatted at 2026-08-11 11:14:54 on dist-test-slave-69rg"
server_key: "0449d63fc2459d7c469483cd9a3b32f9"
server_key_iv: "1f2175ebaf8d886d21f31a2588783504"
server_key_version: "encryptionkey@0"
I20260811 11:14:54.958041 23224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:54.974000 23224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:54.976614 23224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:54.976811 23224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:54.981550 23224 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:37341
I20260811 11:14:54.981592 23300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:37341 every 8 connection(s)
I20260811 11:14:54.982014 23224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:54.982952 23301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:54.985296 23301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737: Bootstrap starting.
I20260811 11:14:54.985954 23301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:54.986271 23301 log.cc:824] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:54.987052 23301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737: No bootstrap required, opened a new log
I20260811 11:14:54.988723 23301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } }
I20260811 11:14:54.988865 23301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:54.988888 23301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6209e7eb25bc4ba18ea03c9ba6b33737, State: Initialized, Role: FOLLOWER
I20260811 11:14:54.989046 23301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } }
I20260811 11:14:54.989113 23301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:54.989135 23301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:54.989204 23301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:54.990113 23301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } }
I20260811 11:14:54.990211 23301 leader_election.cc:302] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6209e7eb25bc4ba18ea03c9ba6b33737; no voters:
I20260811 11:14:54.990413 23301 leader_election.cc:288] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:54.990473 23305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:54.990679 23305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [term 1 LEADER]: Becoming Leader. State: Replica: 6209e7eb25bc4ba18ea03c9ba6b33737, State: Running, Role: LEADER
I20260811 11:14:54.990703 23301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:54.990844 23305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } }
I20260811 11:14:54.991272 23307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6209e7eb25bc4ba18ea03c9ba6b33737. Latest consensus state: current_term: 1 leader_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } } }
I20260811 11:14:54.991253 23306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6209e7eb25bc4ba18ea03c9ba6b33737" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 37341 } } }
I20260811 11:14:54.991413 23307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:54.991420 23306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:54.991917 23316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:54.992260 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23224
I20260811 11:14:54.992389 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:54.992496 23316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:54.992609 30276 external_mini_cluster.cc:1428] Setting key 2e63fc15e86fb7566cbea9e7b01118d3
I20260811 11:14:54.994266 23316 catalog_manager.cc:1347] Generated new cluster ID: ca9c1aff337b4ea68bac770d797caf7b
I20260811 11:14:54.994324 23316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:54.999089 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:54.993236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43403 (local address 127.29.145.62:37341)
0811 11:14:54.993442 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:54.993451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:54.993616 (+ 165us) server_negotiation.cc:407] Connection header received
0811 11:14:54.993792 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:54.993818 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:54.993906 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:54.994297 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:54.994410 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:54.995778 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:54.996871 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:54.997454 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:54.997871 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:54.998336 (+ 465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:54.998340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:54.998340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:54.998531 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:54.998564 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:54.998577 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:54.998784 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:54.998820 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 11:14:54.998879 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260811 11:14:55.000087 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:55.005676 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:55.002077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44739 (local address 127.29.145.62:37341)
0811 11:14:55.002109 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:55.002111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:55.002404 (+ 293us) server_negotiation.cc:407] Connection header received
0811 11:14:55.002458 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:55.002459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:55.002469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:55.002524 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:55.002669 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:55.003368 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:55.004387 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:55.004693 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:55.004752 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:55.005444 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:55.005446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:55.005446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:55.005451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:55.005465 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:55.005469 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:55.005520 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:55.005581 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:14:55.005605 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260811 11:14:55.017984 23316 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:55.018571 23316 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:55.021905 23316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6209e7eb25bc4ba18ea03c9ba6b33737: Generated new TSK 0
I20260811 11:14:55.022110 23316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:55.025190 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23224
2026-08-11T11:14:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:55 dist-test-slave-69rg krb5kdc[23338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:55 dist-test-slave-69rg krb5kdc[23338](info): set up 2 sockets
Aug 11 11:14:55 dist-test-slave-69rg krb5kdc[23338](info): commencing operation
krb5kdc: starting...
W20260811 11:14:56.159411 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:56 dist-test-slave-69rg krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:56.288652 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39249
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:34927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:39249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:56.371935 23353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:56.372149 23353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:56.372174 23353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:56.373250 23353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:56.373289 23353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:56.373303 23353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:56.373315 23353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:56.373328 23353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:56.375428 23353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:34927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:39249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:39249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:56.375694 23353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:56.375941 23353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:56.380971 23360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:56.380990 23362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:56.381008 23359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:56.381008 23353 server_base.cc:1034] running on GCE node
I20260811 11:14:56.381503 23353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:56.381803 23353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:56.383040 23353 hybrid_clock.cc:648] HybridClock initialized: now 1786446896383018 us; error 40 us; skew 500 ppm
Aug 11 11:14:56 dist-test-slave-69rg krb5kdc[23338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446896, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:56.385998 23353 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:56.386595 23353 webserver.cc:460] Webserver started at http://127.29.145.62:38431/ using document root <none> and password file <none>
I20260811 11:14:56.386881 23353 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:56.386929 23353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:56.387024 23353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:56.388078 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "d7b9907cd80e46239fcdbf97d6b0d554"
format_stamp: "Formatted at 2026-08-11 11:14:56 on dist-test-slave-69rg"
server_key: "1fcac1eabee40ebb603b6822d06f62dc"
server_key_iv: "7995a59524a5921373ef98168413b1cd"
server_key_version: "encryptionkey@0"
I20260811 11:14:56.388392 23353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "d7b9907cd80e46239fcdbf97d6b0d554"
format_stamp: "Formatted at 2026-08-11 11:14:56 on dist-test-slave-69rg"
server_key: "1fcac1eabee40ebb603b6822d06f62dc"
server_key_iv: "7995a59524a5921373ef98168413b1cd"
server_key_version: "encryptionkey@0"
I20260811 11:14:56.389732 23353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 11:14:56.390573 23370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:56.390810 23353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:14:56.390892 23353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "d7b9907cd80e46239fcdbf97d6b0d554"
format_stamp: "Formatted at 2026-08-11 11:14:56 on dist-test-slave-69rg"
server_key: "1fcac1eabee40ebb603b6822d06f62dc"
server_key_iv: "7995a59524a5921373ef98168413b1cd"
server_key_version: "encryptionkey@0"
I20260811 11:14:56.390987 23353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:56.394462 23353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:56.396454 23353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:56.396595 23353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:56.401160 23353 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:39249
I20260811 11:14:56.401290 23430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:39249 every 8 connection(s)
I20260811 11:14:56.401595 23353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:56.402768 23431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:56.403944 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23353
I20260811 11:14:56.404074 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:56.404300 30276 external_mini_cluster.cc:1428] Setting key 35e0ebc094ce24914a114208fa4548f6
I20260811 11:14:56.405776 23431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554: Bootstrap starting.
I20260811 11:14:56.406699 23431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:56.407018 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:56.404880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49757 (local address 127.29.145.62:39249)
0811 11:14:56.405063 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:56.405071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:56.405104 (+ 33us) server_negotiation.cc:407] Connection header received
0811 11:14:56.405542 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:56.405549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:56.405573 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:56.405702 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:56.406479 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:56.406483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:56.406483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:56.406488 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:56.406541 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:56.406553 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:56.406663 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:56.406797 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 11:14:56.406845 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260811 11:14:56.407095 23431 log.cc:824] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:56.408044 23431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554: No bootstrap required, opened a new log
I20260811 11:14:56.408334 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:56.410735 23431 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } }
I20260811 11:14:56.410902 23431 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:14:56.410950 23431 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b9907cd80e46239fcdbf97d6b0d554, State: Initialized, Role: FOLLOWER
I20260811 11:14:56.411079 23431 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } }
I20260811 11:14:56.411324 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:56.410024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37907 (local address 127.29.145.62:39249)
0811 11:14:56.410079 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:56.410081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:56.410226 (+ 145us) server_negotiation.cc:407] Connection header received
0811 11:14:56.410298 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:56.410299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:56.410305 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:56.410370 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:56.411042 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:56.411044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:56.411044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:14:56.411050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:56.411067 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:14:56.411072 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:56.411104 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:56.411232 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 11:14:56.411249 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:14:56.411422 23431 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:14:56.411494 23431 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:14:56.411543 23431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:56.412284 23431 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } }
I20260811 11:14:56.412382 23431 leader_election.cc:302] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7b9907cd80e46239fcdbf97d6b0d554; no voters:
I20260811 11:14:56.412559 23431 leader_election.cc:288] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:14:56.412606 23444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:14:56.412789 23444 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [term 1 LEADER]: Becoming Leader. State: Replica: d7b9907cd80e46239fcdbf97d6b0d554, State: Running, Role: LEADER
I20260811 11:14:56.412844 23431 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:56.412940 23444 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } }
I20260811 11:14:56.413349 23446 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7b9907cd80e46239fcdbf97d6b0d554. Latest consensus state: current_term: 1 leader_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } } }
I20260811 11:14:56.413342 23445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7b9907cd80e46239fcdbf97d6b0d554" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 39249 } } }
I20260811 11:14:56.413456 23446 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:56.413463 23445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:56.414135 23453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:56.414774 23453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:56.416497 23453 catalog_manager.cc:1347] Generated new cluster ID: a2838509a0a84edcab12f12578bbad26
I20260811 11:14:56.416548 23453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:56.428638 23453 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:56.428886 23453 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:56.440734 23453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7b9907cd80e46239fcdbf97d6b0d554: Generated new TSK 0
I20260811 11:14:56.440960 23453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:14:56.451442 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23353
2026-08-11T11:14:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1426 ms)
[----------] 18 tests from AuthTokenIssuingTest (25984 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:56 dist-test-slave-69rg krb5kdc[23470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:56 dist-test-slave-69rg krb5kdc[23470](info): set up 2 sockets
Aug 11 11:14:56 dist-test-slave-69rg krb5kdc[23470](info): commencing operation
krb5kdc: starting...
W20260811 11:14:57.562160 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:14:57.689220 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43093
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:14:57.772454 23483 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:14:57.772728 23483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:57.772765 23483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:57.772799 23483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:57.774005 23483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:57.774066 23483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:57.774089 23483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:57.774127 23483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:57.774158 23483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:57.775648 23483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:43093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:57.775928 23483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:57.776161 23483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:57.781338 23492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:57.781481 23483 server_base.cc:1034] running on GCE node
W20260811 11:14:57.781338 23490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:57.781230 23489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:57.781935 23483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:57.782248 23483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:57.783394 23483 hybrid_clock.cc:648] HybridClock initialized: now 1786446897783375 us; error 44 us; skew 500 ppm
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:57.790886 23483 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:14:57.791316 23483 webserver.cc:460] Webserver started at http://127.29.145.62:46607/ using document root <none> and password file <none>
I20260811 11:14:57.791512 23483 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:57.791554 23483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:57.791646 23483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:57.792694 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "31601a041f624a8fa58cdc3155c065f7"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "0935864f6149684b76c6bae438abb620"
server_key_iv: "2ed7a8bbf997a56ba56cc4697e1a976f"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.793023 23483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "31601a041f624a8fa58cdc3155c065f7"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "0935864f6149684b76c6bae438abb620"
server_key_iv: "2ed7a8bbf997a56ba56cc4697e1a976f"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.794448 23483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:14:57.795212 23503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:57.795426 23483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:57.795540 23483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "31601a041f624a8fa58cdc3155c065f7"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "0935864f6149684b76c6bae438abb620"
server_key_iv: "2ed7a8bbf997a56ba56cc4697e1a976f"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.795615 23483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:57.805459 23483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:57.807492 23483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:57.807654 23483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:57.811740 23483 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:43093
I20260811 11:14:57.811841 23564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:43093 every 8 connection(s)
I20260811 11:14:57.812158 23483 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:14:57.813189 23565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:57.814488 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23483
I20260811 11:14:57.814608 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:14:57.814802 30276 external_mini_cluster.cc:1428] Setting key 231fac654b6342615cec90ce12819c0a
I20260811 11:14:57.814971 23565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.61@KRBTEST.COM; defaulting to no policy
I20260811 11:14:57.825927 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.815677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43093 (local address 127.0.0.1:50230)
0811 11:14:57.815863 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.815904 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:57.816423 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:57.816792 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:57.816795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:57.817162 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:57.817393 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.817399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.818537 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.818538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.819350 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.819352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.820127 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.820129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.820218 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.820749 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:57.820778 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:57.822276 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:57.824338 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.824345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.824347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.824534 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.824821 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.824822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.824823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.824909 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.825275 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:57.825280 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:57.825350 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 11:14:57.825450 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 11:14:57.825711 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":117,"thread_start_us":90,"threads_started":1}
I20260811 11:14:57.825932 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.815705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.29.145.62:43093)
0811 11:14:57.815889 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:57.815898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:57.816012 (+ 114us) server_negotiation.cc:407] Connection header received
0811 11:14:57.816639 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:57.816642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:57.816664 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:57.816762 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:57.817499 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.818433 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.819445 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.819989 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.820111 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.822366 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:57.822374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:57.822374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:57.822381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:57.824245 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.824656 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.824659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.824660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.824745 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.824986 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.824987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.824987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.825201 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:57.825294 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:57.825485 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 11:14:57.825630 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
I20260811 11:14:57.826946 23565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:14:57.827366 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.827206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52660)
0811 11:14:57.827302 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.827347 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.61@KRBTEST.COM" created.
W20260811 11:14:57.827546 23505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
I20260811 11:14:57.827994 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.827894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52676)
0811 11:14:57.827939 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.827970 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:14:57.828174 23565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:35327: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
I20260811 11:14:57.845511 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:35327
--webserver_interface=127.29.145.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:14:57.877409 23565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } attempt: 1
I20260811 11:14:57.877938 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.877779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52684)
0811 11:14:57.877870 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.877907 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 11:14:57.878297 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.878201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52686)
0811 11:14:57.878248 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.878273 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 11:14:57.878386 23565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:35327: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
W20260811 11:14:57.930620 23572 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:14:57.930866 23572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:57.930899 23572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:57.930944 23572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:57.932055 23572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:57.932120 23572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:57.932147 23572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:57.932170 23572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:57.932196 23572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:57.933737 23572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:35327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.145.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:57.934031 23572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:57.934263 23572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:57.939184 23580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:57.939246 23577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:57.939277 23578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:57.939384 23572 server_base.cc:1034] running on GCE node
I20260811 11:14:57.939620 23572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:57.939951 23572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:57.941159 23572 hybrid_clock.cc:648] HybridClock initialized: now 1786446897941130 us; error 46 us; skew 500 ppm
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:57.943822 23572 init.cc:377] Logged in from keytab as kudu/127.29.145.61@KRBTEST.COM (short username kudu)
I20260811 11:14:57.944347 23572 webserver.cc:460] Webserver started at http://127.29.145.61:35691/ using document root <none> and password file <none>
I20260811 11:14:57.944598 23572 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:57.944669 23572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:57.944823 23572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:14:57.945955 23572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data/instance:
uuid: "f287a43461b343098693853d1165e96e"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "4f5bd6fa209539d3867d7bb46a34e3d9"
server_key_iv: "b9012d892e115e3db73309206ea38022"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.946331 23572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal/instance:
uuid: "f287a43461b343098693853d1165e96e"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "4f5bd6fa209539d3867d7bb46a34e3d9"
server_key_iv: "b9012d892e115e3db73309206ea38022"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.946503 23565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } attempt: 2
I20260811 11:14:57.946895 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.946768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52694)
0811 11:14:57.946831 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.946866 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:14:57.947248 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.947153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52710)
0811 11:14:57.947200 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.947223 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:14:57.947333 23565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:35327: Network error: Client connection negotiation failed: client connection to 127.29.145.61:35327: connect: Connection refused (error 111)
I20260811 11:14:57.948009 23572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:14:57.948846 23590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:57.949074 23572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:14:57.949182 23572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "f287a43461b343098693853d1165e96e"
format_stamp: "Formatted at 2026-08-11 11:14:57 on dist-test-slave-69rg"
server_key: "4f5bd6fa209539d3867d7bb46a34e3d9"
server_key_iv: "b9012d892e115e3db73309206ea38022"
server_key_version: "encryptionkey@0"
I20260811 11:14:57.949270 23572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:57.964388 23572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:57.966430 23572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:57.966596 23572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:57.970878 23572 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.61:35327
I20260811 11:14:57.970933 23647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.61:35327 every 8 connection(s)
I20260811 11:14:57.971304 23572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
I20260811 11:14:57.972306 23649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:57.973937 23649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:57.981554 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23572
I20260811 11:14:57.981676 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-1/wal/instance
I20260811 11:14:57.981902 30276 external_mini_cluster.cc:1428] Setting key 6571fcd00abf13f9ac57519e401ec9f3
I20260811 11:14:57.982990 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.974638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.29.145.62:43093)
0811 11:14:57.974694 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:57.974696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:57.974923 (+ 227us) server_negotiation.cc:407] Connection header received
0811 11:14:57.975437 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:57.975438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:57.975455 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:57.975519 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:57.976126 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.976618 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.977618 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.977894 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.977974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.980163 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:57.980165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:57.980165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:57.980173 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:57.981402 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.981957 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.981959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.981959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.982030 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.982340 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.982341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.982341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.982451 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:57.982508 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:57.982850 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 11:14:57.982925 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:14:57.983271 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.974577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43093 (local address 127.0.0.1:50242)
0811 11:14:57.974798 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.974835 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:57.975327 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:57.975585 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:57.975589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:57.975838 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:57.976024 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.976029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.976727 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.976728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.977521 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.977523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.977995 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.977996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.978148 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.978588 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:57.978596 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:57.980063 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:57.981489 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.981498 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.981500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.981861 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.982109 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.982110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.982111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.982266 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.982518 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:57.982525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:57.982712 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 11:14:57.982830 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 11:14:57.983009 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":119,"threads_started":1}
I20260811 11:14:57.983976 23649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:57 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.60@KRBTEST.COM; defaulting to no policy
I20260811 11:14:57.991724 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.984272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52726)
0811 11:14:57.984323 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.984334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:57.984382 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:57.984637 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:57.984638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:57.984781 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:57.984825 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.984826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.985671 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.985672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.986241 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:57.986244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.986939 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.986939 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:57.986994 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.987405 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:57.987408 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:57.988508 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:57.990624 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.990625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.990625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.990784 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.991019 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:57.991019 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:57.991020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:57.991073 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:57.991408 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:57.991409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:57.991450 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:14:57.991519 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 11:14:57.991663 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 11:14:57.991829 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.984337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.29.145.61:35327)
0811 11:14:57.984507 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:57.984511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:57.984526 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:14:57.984543 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:57.984544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:57.984553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:57.984637 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:57.984898 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.985574 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.986339 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:57.986860 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:57.986942 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:57.988594 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:57.988602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:57.988602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:57.988609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:57.990525 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.990873 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.990877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.990877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.990943 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:57.991143 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:57.991144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:57.991144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:57.991321 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:57.991388 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:57.991577 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 11:14:57.991713 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260811 11:14:57.992425 23649 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:14:57.992839 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.992708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55550)
0811 11:14:57.992759 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.992812 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 11:14:57.993013 23593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
I20260811 11:14:57.993485 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:57.993376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55556)
0811 11:14:57.993420 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:57.993446 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:14:57.993665 23649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:37771: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
I20260811 11:14:58.009182 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:37771
--webserver_interface=127.29.145.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:14:58.038496 23565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } attempt: 3
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:14:58.042870 23649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } attempt: 1
I20260811 11:14:58.043272 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.043145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55572)
0811 11:14:58.043201 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.043235 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 11:14:58.043656 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.043552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55582)
0811 11:14:58.043601 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.043626 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 11:14:58.043819 23649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:37771: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
I20260811 11:14:58.046244 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.038920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52734)
0811 11:14:58.038979 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.038998 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.039070 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.039228 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.039229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.039417 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.039484 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.039485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.040196 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.040197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.040748 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.040749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.041144 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.041145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.041234 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.041730 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.041735 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.043341 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.045270 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.045271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.045271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.045442 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.045652 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.045653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.045653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.045708 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.045993 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.045994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.046037 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.046092 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.046173 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 11:14:58.046304 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.038997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.29.145.61:35327)
0811 11:14:58.039074 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.039077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.039098 (+ 21us) server_negotiation.cc:407] Connection header received
0811 11:14:58.039130 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.039131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.039145 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.039222 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.039582 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.040098 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.040841 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.041067 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.041147 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.043465 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.043467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.043467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.043475 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.045180 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.045515 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.045516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.045517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.045580 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.045771 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.045772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.045772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.045927 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.045997 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.046138 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.046222 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:14:58.046720 23565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:14:58.047070 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.046964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55592)
0811 11:14:58.047012 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.047038 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 11:14:58.047435 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.047375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55608)
0811 11:14:58.047399 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.047420 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260811 11:14:58.047535 23565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:37771: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
I20260811 11:14:58.078693 23565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } attempt: 1
I20260811 11:14:58.079170 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.079031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55614)
0811 11:14:58.079096 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.079135 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:14:58.079522 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.079415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55616)
0811 11:14:58.079463 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.079490 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 11:14:58.079620 23565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:37771: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
W20260811 11:14:58.098392 23658 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:14:58.098637 23658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:14:58.098666 23658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:14:58.098711 23658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:14:58.099865 23658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:14:58.099928 23658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:14:58.099953 23658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:14:58.099982 23658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:14:58.100010 23658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:14:58.101519 23658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:43093,127.29.145.61:35327,127.29.145.60:37771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:37771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.145.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:14:58.101804 23658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:14:58.102093 23658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:14:58.106961 23665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:58.106971 23667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:14:58.106971 23664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:14:58.107312 23658 server_base.cc:1034] running on GCE node
I20260811 11:14:58.107548 23658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:14:58.107889 23658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:14:58.109069 23658 hybrid_clock.cc:648] HybridClock initialized: now 1786446898109058 us; error 58 us; skew 500 ppm
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446898, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:14:58.111800 23658 init.cc:377] Logged in from keytab as kudu/127.29.145.60@KRBTEST.COM (short username kudu)
I20260811 11:14:58.112007 23649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } attempt: 2
I20260811 11:14:58.112299 23658 webserver.cc:460] Webserver started at http://127.29.145.60:44537/ using document root <none> and password file <none>
I20260811 11:14:58.112443 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.112298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55630)
0811 11:14:58.112373 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.112412 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 11:14:58.112584 23658 fs_manager.cc:359] Metadata directory not provided
I20260811 11:14:58.112686 23658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:14:58.112820 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.112731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55640)
0811 11:14:58.112764 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.112791 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260811 11:14:58.112850 23658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
W20260811 11:14:58.112910 23649 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:37771: Network error: Client connection negotiation failed: client connection to 127.29.145.60:37771: connect: Connection refused (error 111)
I20260811 11:14:58.114073 23658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data/instance:
uuid: "c41646faaf5b46e38a1ea1f1d2574324"
format_stamp: "Formatted at 2026-08-11 11:14:58 on dist-test-slave-69rg"
server_key: "136ca9d55b55c29e89a5e7e8b79e115e"
server_key_iv: "cf8a0966ecdbc6010a5bb26e8666eb02"
server_key_version: "encryptionkey@0"
I20260811 11:14:58.114444 23658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal/instance:
uuid: "c41646faaf5b46e38a1ea1f1d2574324"
format_stamp: "Formatted at 2026-08-11 11:14:58 on dist-test-slave-69rg"
server_key: "136ca9d55b55c29e89a5e7e8b79e115e"
server_key_iv: "cf8a0966ecdbc6010a5bb26e8666eb02"
server_key_version: "encryptionkey@0"
I20260811 11:14:58.115783 23658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:14:58.116490 23674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:14:58.116683 23658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 11:14:58.116755 23658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "c41646faaf5b46e38a1ea1f1d2574324"
format_stamp: "Formatted at 2026-08-11 11:14:58 on dist-test-slave-69rg"
server_key: "136ca9d55b55c29e89a5e7e8b79e115e"
server_key_iv: "cf8a0966ecdbc6010a5bb26e8666eb02"
server_key_version: "encryptionkey@0"
I20260811 11:14:58.116840 23658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:14:58.126535 23658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:14:58.128432 23658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:14:58.128577 23658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:14:58.132545 23658 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.60:37771
I20260811 11:14:58.132653 23738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.60:37771 every 8 connection(s)
I20260811 11:14:58.132956 23658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
I20260811 11:14:58.133898 23739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:14:58.134371 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23658
I20260811 11:14:58.134464 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786446611237378-30276-0/minicluster-data/master-2/wal/instance
I20260811 11:14:58.134668 30276 external_mini_cluster.cc:1428] Setting key 394683ff717fe8b4a38fcdc29db43b74
I20260811 11:14:58.135815 23739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446898, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:14:58.149217 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.136738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.29.145.62:43093)
0811 11:14:58.136794 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.136796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.136910 (+ 114us) server_negotiation.cc:407] Connection header received
0811 11:14:58.136980 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.136981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.136990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.137065 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.137521 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.138276 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.139584 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.139869 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.139950 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.142619 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.142621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.142621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.142630 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.147276 (+ 4646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.147804 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.147806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.147806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.147873 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.148325 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.148326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.148326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.148521 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.148573 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.149010 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.149107 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:14:58.149436 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.136627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:43093 (local address 127.0.0.1:50258)
0811 11:14:58.136790 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.136835 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.136914 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.137095 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.137097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.137337 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.137429 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.137433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.138364 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.138365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.139498 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.139501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.140028 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.140029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.140115 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.140684 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.140695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.142506 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.147380 (+ 4874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.147388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.147391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.147717 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.148090 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.148091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.148092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.148239 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.148688 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.148693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.148874 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.148971 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.149197 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260811 11:14:58.149436 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.135206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.29.145.60:37771)
0811 11:14:58.135377 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.135384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.135409 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:14:58.135995 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.136002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.136050 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.136163 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.137060 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.138384 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.139467 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.139933 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.140033 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.143111 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.143118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.143118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.143124 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.147540 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.148109 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.148112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.148113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.148239 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.148496 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.148497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.148497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.148879 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.149017 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.149179 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.149246 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260811 11:14:58.150499 23739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446898, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:14:58.154775 23565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } attempt: 2
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:14:58.158012 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.150828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:35327 (local address 127.0.0.1:52740)
0811 11:14:58.150872 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.150882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.150919 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.151092 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.151093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.151217 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.151268 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.151269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.151940 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.151941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.152505 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.152507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.152961 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.152962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.153016 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.153551 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.153557 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.154955 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.156866 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.156867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.156868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.157067 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.157293 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.157294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.157294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.157358 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.157666 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.157667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.157709 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.157781 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.157913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 11:14:58.158044 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.150893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.29.145.61:35327)
0811 11:14:58.150945 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.150948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.150956 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:14:58.150985 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.150986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.150997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.151068 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.151348 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.151858 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.152622 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.152876 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.152943 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.155042 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.155043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.155044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.155052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.156773 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.157143 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.157145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.157145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.157216 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.157432 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.157433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.157434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.157579 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.157667 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.157849 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.157935 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:14:58.158805 23739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:14:58.159997 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.150676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.29.145.62:43093)
0811 11:14:58.150719 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.150721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.150731 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:58.150765 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.150766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.150775 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.150840 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.151446 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.151911 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.152590 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.152806 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.152857 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.156036 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.156038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.156038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.156045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.157867 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.158273 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.158274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.158275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.158777 (+ 502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.158990 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.158991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.158991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.159137 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.159302 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.159800 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.159877 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:14:58.162604 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.155017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55656)
0811 11:14:58.155059 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.155070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.155105 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.155543 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.155544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.155708 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.155752 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.155754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.156589 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.156591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.157425 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.157428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.157909 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.157911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.157968 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.158351 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.158355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.159701 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.161537 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.161538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.161539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.161756 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.161964 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.161964 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.161964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.162028 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.162299 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.162301 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.162347 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.162423 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.162512 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 11:14:58.162636 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.155386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.29.145.60:37771)
0811 11:14:58.155440 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.155442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.155451 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:58.155476 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.155477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.155484 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.155531 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.155842 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.156515 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.157492 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.157789 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.158133 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.159778 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.159779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.159780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.159786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.161453 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.161842 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.161843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.161843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.161901 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.162094 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.162095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.162095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.162222 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.162286 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.162461 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.162535 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 11:14:58.164327 23565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: Bootstrap starting.
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446898, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:14:58.165126 23565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: Neither blocks nor log segments found. Creating new log.
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:14:58.165493 23565 log.cc:824] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:58.166471 23565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: No bootstrap required, opened a new log
I20260811 11:14:58.167994 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.159054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55672)
0811 11:14:58.159110 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.159122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.159170 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.159431 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.159432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.159563 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.159588 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.159589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.161740 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.161741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.162519 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.162521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.162980 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.162981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.163028 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.163420 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.163425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.164849 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.166938 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.166939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.166939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.167131 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.167327 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.167328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.167328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.167385 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.167698 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.167700 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.167738 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.167805 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.167890 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:14:58.168392 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.159143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.29.145.60:37771)
0811 11:14:58.159296 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.159298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.159304 (+ 6us) server_negotiation.cc:407] Connection header received
0811 11:14:58.159321 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.159322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.159331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.159402 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.160100 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.161663 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.162596 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.162895 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.163159 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.164943 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.164945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.164945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.164952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.166845 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.167197 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.167198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.167198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.167237 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.167458 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.167459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.167459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.167616 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.167695 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.168177 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.168249 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260811 11:14:58.168828 23565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.169008 23565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31601a041f624a8fa58cdc3155c065f7, State: Initialized, Role: FOLLOWER
I20260811 11:14:58.169094 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.160688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.29.145.61:35327)
0811 11:14:58.160728 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.160729 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.160737 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:14:58.160787 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.160788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.160816 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.160867 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.161324 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.161997 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.162927 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.163254 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.163299 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.166000 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.166001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.166002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.166010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.167698 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.168200 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.168201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.168201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.168263 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.168515 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.168516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.168517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.168663 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.168879 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.168921 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.168999 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:14:58.169122 23565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.169456 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.169518 23565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:58.169559 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.170097 23739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: Bootstrap starting.
I20260811 11:14:58.170873 23739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:58.171142 23739 log.cc:824] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: Log is configured to *not* fsync() on all Append() calls
W20260811 11:14:58.171741 23761 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:58.171846 23761 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:58.172139 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:14:58.172168 23739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: No bootstrap required, opened a new log
I20260811 11:14:58.173159 23534 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "31601a041f624a8fa58cdc3155c065f7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50272
I20260811 11:14:58.173274 23534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 11:14:58.173317 23534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:58.174252 23534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.174289 23739 raft_consensus.cc:348] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.174412 23739 raft_consensus.cc:729] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41646faaf5b46e38a1ea1f1d2574324, State: Initialized, Role: FOLLOWER
I20260811 11:14:58.174481 23534 leader_election.cc:288] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [CANDIDATE]: Term 1 election: Requested vote from peers f287a43461b343098693853d1165e96e (127.29.145.61:35327), c41646faaf5b46e38a1ea1f1d2574324 (127.29.145.60:37771)
I20260811 11:14:58.174484 23739 consensus_queue.cc:260] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.175156 23739 sys_catalog.cc:564] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:58.172436 23483 master_runner.cc:186] Error getting master registration for 127.29.145.61:35327: OK, instance_id { permanent_uuid: "f287a43461b343098693853d1165e96e" instance_seqno: 1786446897969598 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:14:58.175436 23483 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:14:58.175643 23765 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.175750 23711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31601a041f624a8fa58cdc3155c065f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c41646faaf5b46e38a1ea1f1d2574324"
I20260811 11:14:58.175869 23711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:58.176725 23711 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31601a041f624a8fa58cdc3155c065f7 in term 1.
I20260811 11:14:58.176983 23506 leader_election.cc:302] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31601a041f624a8fa58cdc3155c065f7, c41646faaf5b46e38a1ea1f1d2574324; no voters:
I20260811 11:14:58.177151 23748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 11:14:58.177712 23783 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:58.177798 23783 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:58.177351 23748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [term 1 LEADER]: Becoming Leader. State: Replica: 31601a041f624a8fa58cdc3155c065f7, State: Running, Role: LEADER
I20260811 11:14:58.177937 23748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.178421 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31601a041f624a8fa58cdc3155c065f7. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.178524 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:58.178771 23787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:14:58.175751 23765 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.179541 23787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:14:58.181193 23711 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [term 1 FOLLOWER]: Refusing update from remote peer 31601a041f624a8fa58cdc3155c065f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:58.181555 23784 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:58.180469 23658 master_runner.cc:186] Error getting master registration for 127.29.145.61:35327: OK, instance_id { permanent_uuid: "f287a43461b343098693853d1165e96e" instance_seqno: 1786446897969598 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:14:58.182827 23658 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:14:58.183666 23765 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31601a041f624a8fa58cdc3155c065f7. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.183749 23765 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.191480 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.179291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.29.145.61:35327)
0811 11:14:58.179336 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.179338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.182543 (+ 3205us) server_negotiation.cc:407] Connection header received
0811 11:14:58.182661 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.182662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.182673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.182733 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.184217 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.184961 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.186141 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.186473 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.186532 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.188243 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.188245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.188245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.188253 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.190055 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.190514 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.190515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.190516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.190576 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.190813 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.190814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.190815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.190960 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.191041 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.191290 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.191373 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 11:14:58.193248 23784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.193362 23784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:58.193447 23791 mvcc.cc:205] Tried to move back new op lower bound from 7317286494948028416 to 7317286494937759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317286494948028416}]
I20260811 11:14:58.194078 23792 mvcc.cc:205] Tried to move back new op lower bound from 7317286494948028416 to 7317286494937759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317286494948028416}]
I20260811 11:14:58.194314 23765 sys_catalog.cc:455] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.194394 23765 sys_catalog.cc:458] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.194867 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.179760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.29.145.60:37771)
0811 11:14:58.179796 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.179798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.183436 (+ 3638us) server_negotiation.cc:407] Connection header received
0811 11:14:58.184306 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.184307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.184318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.184347 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.185081 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.185871 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.186905 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.187225 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.187264 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.190013 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.190015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.190015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.190023 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.191870 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.192508 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.192509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.192510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.192565 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.193748 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.193749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.193749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.193980 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.194061 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.194667 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.194752 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 11:14:58.195572 23787 catalog_manager.cc:1347] Generated new cluster ID: c02f1180bbcc454e99334babd362dfdb
I20260811 11:14:58.195622 23787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:14:58.197863 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.179559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50282 (local address 127.29.145.62:43093)
0811 11:14:58.179599 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.179601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.182393 (+ 2792us) server_negotiation.cc:407] Connection header received
0811 11:14:58.182470 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.182471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.182483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.182539 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.190993 (+ 8454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.191717 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.192748 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.193023 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.193092 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.194579 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.194581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.194581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.194588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.196351 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.196739 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.196740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.196741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.196803 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.196995 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.196996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.196996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.197116 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.197227 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.197616 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.197729 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:14:58.203974 23649 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } attempt: 3
Aug 11 11:14:58 dist-test-slave-69rg krb5kdc[23470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446897, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:14:58.211946 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.204191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:37771 (local address 127.0.0.1:55678)
0811 11:14:58.204325 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 11:14:58.204335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:14:58.204379 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:14:58.204577 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:14:58.204579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:14:58.204759 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:14:58.204809 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.204811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.205681 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.205682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.206520 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:14:58.206522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.207032 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.207033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:14:58.207092 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.207526 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:14:58.207531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:14:58.209004 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:14:58.210907 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.210908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.210909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.211117 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.211324 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:14:58.211325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:14:58.211325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:14:58.211378 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:14:58.211644 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:14:58.211645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:14:58.211689 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 11:14:58.211752 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:14:58.211847 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 11:14:58.212289 23746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:58.204273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55678 (local address 127.29.145.60:37771)
0811 11:14:58.204317 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.204319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.204366 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:14:58.204435 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.204435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.204447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.204526 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.204892 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.205588 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.206619 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:58.206950 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:58.207019 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:58.209086 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:14:58.209087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:14:58.209087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:14:58.209094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:14:58.210814 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.211206 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.211207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.211208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.211259 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:14:58.211441 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:14:58.211442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:14:58.211442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:14:58.211563 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:14:58.211618 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:58.212116 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 11:14:58.212204 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 11:14:58.213708 23649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: Bootstrap starting.
I20260811 11:14:58.214447 23649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: Neither blocks nor log segments found. Creating new log.
I20260811 11:14:58.214709 23649 log.cc:824] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:14:58.216377 23649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: No bootstrap required, opened a new log
I20260811 11:14:58.218263 23649 raft_consensus.cc:348] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.218390 23649 raft_consensus.cc:729] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f287a43461b343098693853d1165e96e, State: Initialized, Role: FOLLOWER
I20260811 11:14:58.218454 23649 consensus_queue.cc:260] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } }
I20260811 11:14:58.218778 23649 sys_catalog.cc:564] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:14:58.219621 23620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:14:58.220547 23620 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [term 1 FOLLOWER]: Refusing update from remote peer 31601a041f624a8fa58cdc3155c065f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:14:58.220741 23621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31601a041f624a8fa58cdc3155c065f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f287a43461b343098693853d1165e96e"
I20260811 11:14:58.221386 23621 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31601a041f624a8fa58cdc3155c065f7 in term 1.
I20260811 11:14:58.222249 23748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:14:58.225093 23809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.225250 23809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.225343 23809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31601a041f624a8fa58cdc3155c065f7. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.225425 23809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.225484 23809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.225564 23809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:14:58.226209 23810 mvcc.cc:205] Tried to move back new op lower bound from 7317286494948028416 to 7317286494937759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317286494948028416}]
W20260811 11:14:58.226897 23808 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:14:58.227479 23808 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:14:58.233067 23748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "31601a041f624a8fa58cdc3155c065f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31601a041f624a8fa58cdc3155c065f7" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 43093 } } peers { permanent_uuid: "f287a43461b343098693853d1165e96e" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 35327 } } peers { permanent_uuid: "c41646faaf5b46e38a1ea1f1d2574324" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 37771 } } }
I20260811 11:14:58.233193 23748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:14:58.237913 23787 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:14:58.238806 23787 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:14:58.251389 23787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31601a041f624a8fa58cdc3155c065f7: Generated new TSK 0
I20260811 11:14:58.251617 23787 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:14:58.282196 23568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.280533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50288 (local address 127.29.145.62:43093)
0811 11:14:58.280580 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.280583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.281472 (+ 889us) server_negotiation.cc:407] Connection header received
0811 11:14:58.281581 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.281582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.281594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.281627 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.282103 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50288: BlockingRecv error: recv got EOF from 127.0.0.1:50288 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 11:14:58.283154 23746 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.280886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.29.145.60:37771)
0811 11:14:58.280943 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.280945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.282643 (+ 1698us) server_negotiation.cc:407] Connection header received
0811 11:14:58.282740 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.282741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.282752 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.282781 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.283079 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55680: BlockingRecv error: recv got EOF from 127.0.0.1:55680 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 11:14:58.284124 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.281197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.29.145.61:35327)
0811 11:14:58.281237 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.281239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.283458 (+ 2219us) server_negotiation.cc:407] Connection header received
0811 11:14:58.283624 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.283625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.283637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.283668 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.284052 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52756: BlockingRecv error: recv got EOF from 127.0.0.1:52756 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 11:14:58.286854 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.286062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.29.145.61:35327)
0811 11:14:58.286100 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.286102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.286323 (+ 221us) server_negotiation.cc:407] Connection header received
0811 11:14:58.286438 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.286439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.286458 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.286488 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.286806 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52758: BlockingRecv error: recv got EOF from 127.0.0.1:52758 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 11:14:58.290210 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.289314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52766 (local address 127.29.145.61:35327)
0811 11:14:58.289350 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.289352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.289599 (+ 247us) server_negotiation.cc:407] Connection header received
0811 11:14:58.289699 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.289700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.289710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.289739 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.290151 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52766: BlockingRecv error: recv got EOF from 127.0.0.1:52766 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 11:14:58.294320 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.293506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52778 (local address 127.29.145.61:35327)
0811 11:14:58.293540 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.293542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.293768 (+ 226us) server_negotiation.cc:407] Connection header received
0811 11:14:58.293914 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.293915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.293925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.293955 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.294275 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52778: BlockingRecv error: recv got EOF from 127.0.0.1:52778 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 11:14:58.319415 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.310101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.29.145.61:35327)
0811 11:14:58.310151 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.310153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.318774 (+ 8621us) server_negotiation.cc:407] Connection header received
0811 11:14:58.318871 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.318872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.318888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.318921 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.319342 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52792: BlockingRecv error: recv got EOF from 127.0.0.1:52792 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 11:14:58.335004 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.334131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.29.145.61:35327)
0811 11:14:58.334181 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.334183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.334414 (+ 231us) server_negotiation.cc:407] Connection header received
0811 11:14:58.334500 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.334501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.334515 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.334573 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.334935 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52794: BlockingRecv error: recv got EOF from 127.0.0.1:52794 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 11:14:58.366770 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.358159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.29.145.61:35327)
0811 11:14:58.358220 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.358222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.366081 (+ 7859us) server_negotiation.cc:407] Connection header received
0811 11:14:58.366207 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.366209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.366223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.366256 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.366704 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52798: BlockingRecv error: recv got EOF from 127.0.0.1:52798 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 11:14:58.400983 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.400172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52804 (local address 127.29.145.61:35327)
0811 11:14:58.400252 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.400255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.400383 (+ 128us) server_negotiation.cc:407] Connection header received
0811 11:14:58.400460 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.400461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.400471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.400549 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.400916 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52804: BlockingRecv error: recv got EOF from 127.0.0.1:52804 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 11:14:58.467438 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.466575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.29.145.61:35327)
0811 11:14:58.466693 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.466695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.466813 (+ 118us) server_negotiation.cc:407] Connection header received
0811 11:14:58.466908 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.466909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.466924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.466993 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.467355 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52810: BlockingRecv error: recv got EOF from 127.0.0.1:52810 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 11:14:58.597620 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.596946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.29.145.61:35327)
0811 11:14:58.597026 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.597028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.597084 (+ 56us) server_negotiation.cc:407] Connection header received
0811 11:14:58.597148 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.597149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.597160 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.597214 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.597558 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52812: BlockingRecv error: recv got EOF from 127.0.0.1:52812 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 11:14:58.855722 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:14:58.855027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.29.145.61:35327)
0811 11:14:58.855105 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:58.855107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:58.855116 (+ 9us) server_negotiation.cc:407] Connection header received
0811 11:14:58.855153 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:58.855154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:58.855165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:58.855244 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:14:58.855603 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52820: BlockingRecv error: recv got EOF from 127.0.0.1:52820 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:14:59.178493 23783 catalog_manager.cc:1259] Loaded cluster ID: c02f1180bbcc454e99334babd362dfdb
I20260811 11:14:59.178614 23783 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: loading cluster ID for follower catalog manager: success
I20260811 11:14:59.179670 23783 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: acquiring CA information for follower catalog manager: success
I20260811 11:14:59.180094 23783 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c41646faaf5b46e38a1ea1f1d2574324: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:14:59.228225 23808 catalog_manager.cc:1259] Loaded cluster ID: c02f1180bbcc454e99334babd362dfdb
I20260811 11:14:59.228436 23808 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: loading cluster ID for follower catalog manager: success
I20260811 11:14:59.229568 23808 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: acquiring CA information for follower catalog manager: success
I20260811 11:14:59.230048 23808 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f287a43461b343098693853d1165e96e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:14:59.372874 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:59.369311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.29.145.61:35327)
0811 11:14:59.369490 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:59.369493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:59.369517 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:14:59.369648 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:59.369650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:59.369742 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:59.369894 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:59.370116 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.370914 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.371825 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.372081 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.372169 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:59.372394 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:59.372567 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:59.372677 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:59.372710 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 11:14:59.372739 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":122,"threads_started":1}
I20260811 11:14:59.378973 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:59.375583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42494 (local address 127.29.145.60:37771)
0811 11:14:59.375766 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:59.375769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:59.375782 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:14:59.375910 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:59.375911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:59.375995 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:59.376141 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:59.376377 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.377075 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.378027 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.378266 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.378327 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:59.378543 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:59.378689 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:59.378756 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:59.378810 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 11:14:59.378831 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260811 11:14:59.384723 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:59.381551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.29.145.60:37771)
0811 11:14:59.381599 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:59.381601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:59.381609 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:14:59.381635 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:59.381636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:59.381646 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:59.381700 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:59.382152 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.382742 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.383891 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.384177 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.384235 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:59.384438 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:59.384472 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:59.384502 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:59.384615 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 11:14:59.384636 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:14:59.384745 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:59.381244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.29.145.61:35327)
0811 11:14:59.381320 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:59.381323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:59.381570 (+ 247us) server_negotiation.cc:407] Connection header received
0811 11:14:59.381692 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:59.381693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:59.381705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:59.381764 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:59.382138 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.382961 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.383844 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.384161 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.384235 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:59.384384 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:59.384436 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:59.384487 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:59.384598 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 11:14:59.384631 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:14:59.385402 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:14:59.381289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.29.145.62:43093)
0811 11:14:59.381581 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 11:14:59.381583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:14:59.381593 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:14:59.381650 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:14:59.381651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:14:59.381724 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:14:59.381864 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:14:59.382142 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.382990 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.384063 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:14:59.384401 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:14:59.384469 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:14:59.384964 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:14:59.385099 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:14:59.385176 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:14:59.385262 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 11:14:59.385291 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":75,"threads_started":1}
I20260811 11:14:59.387362 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23483
I20260811 11:14:59.399257 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23572
I20260811 11:14:59.409479 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23658
2026-08-11T11:14:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2959 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:14:59 dist-test-slave-69rg krb5kdc[23915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:14:59 dist-test-slave-69rg krb5kdc[23915](info): set up 2 sockets
Aug 11 11:14:59 dist-test-slave-69rg krb5kdc[23915](info): commencing operation
krb5kdc: starting...
W20260811 11:15:00.544135 30276 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/test-user.keytab.
Aug 11 11:15:00 dist-test-slave-69rg krb5kdc[23915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.29.145.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
Entry for principal kudu/127.29.145.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab.
I20260811 11:15:00.685746 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:40915
--webserver_interface=127.29.145.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:15:00.773437 23929 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:15:00.773681 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:15:00.773700 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:15:00.773722 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:15:00.774922 23929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:15:00.774968 23929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:15:00.774983 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:15:00.774995 23929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:15:00.775008 23929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:15:00.776520 23929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.62.keytab
--principal=kudu/127.29.145.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.62:40915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.29.145.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:15:00.776741 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:15:00.776923 23929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:15:00.782358 23939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:15:00.782512 23937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:15:00.782577 23929 server_base.cc:1034] running on GCE node
W20260811 11:15:00.782696 23936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:15:00.782981 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:15:00.783309 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:15:00.784482 23929 hybrid_clock.cc:648] HybridClock initialized: now 1786446900784469 us; error 30 us; skew 500 ppm
Aug 11 11:15:00 dist-test-slave-69rg krb5kdc[23915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:15:00.787226 23929 init.cc:377] Logged in from keytab as kudu/127.29.145.62@KRBTEST.COM (short username kudu)
I20260811 11:15:00.787699 23929 webserver.cc:460] Webserver started at http://127.29.145.62:40171/ using document root <none> and password file <none>
I20260811 11:15:00.787948 23929 fs_manager.cc:359] Metadata directory not provided
I20260811 11:15:00.788004 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:15:00.788106 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:15:00.789243 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data/instance:
uuid: "f387c9464e274175bb8458eb5059ec27"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "b57250095cfbca7ee3cd61ac44571603"
server_key_iv: "cd6504ffcc5c064e207db0626ae46d09"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.789628 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance:
uuid: "f387c9464e274175bb8458eb5059ec27"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "b57250095cfbca7ee3cd61ac44571603"
server_key_iv: "cd6504ffcc5c064e207db0626ae46d09"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.791100 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:15:00.792043 23946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:15:00.792308 23929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:15:00.792409 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal
uuid: "f387c9464e274175bb8458eb5059ec27"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "b57250095cfbca7ee3cd61ac44571603"
server_key_iv: "cd6504ffcc5c064e207db0626ae46d09"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.792496 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:15:00.802349 23929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:15:00.804598 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:15:00.804754 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:15:00.809361 23929 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.62:40915
I20260811 11:15:00.809397 24006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.62:40915 every 8 connection(s)
I20260811 11:15:00.809800 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/data/info.pb
I20260811 11:15:00.810956 24007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:15:00.811964 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 23929
I20260811 11:15:00.812065 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-0/wal/instance
I20260811 11:15:00.812282 30276 external_mini_cluster.cc:1428] Setting key 9f587a2376d1e054c9e74b866e7d3c29
I20260811 11:15:00.813205 24007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:15:00 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.61@KRBTEST.COM" created.
I20260811 11:15:00.826699 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.814073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.29.145.62:40915)
0811 11:15:00.814309 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:00.814320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:00.814608 (+ 288us) server_negotiation.cc:407] Connection header received
0811 11:15:00.815703 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:00.815706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:00.815747 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:00.815858 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:00.816515 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.817963 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.818920 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.819498 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.819630 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:00.822234 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:00.822243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:00.822243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:00.822253 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:00.824055 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:00.824509 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:00.824512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:00.824513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:00.824599 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:00.825114 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:00.825115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:00.825115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:00.825369 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:00.825453 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:00.826193 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 11:15:00.826362 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":91,"threads_started":1}
I20260811 11:15:00.826892 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.814037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:40915 (local address 127.0.0.1:49026)
0811 11:15:00.814392 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.814434 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:00.815215 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:00.815899 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:00.815902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:00.816195 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:00.816386 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:00.816390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.818115 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.818117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:00.818822 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:00.818824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.819647 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.819648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:00.819712 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:00.820360 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:00.820370 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:00.822131 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:00.824153 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:00.824160 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:00.824162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:00.824420 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:00.824724 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:00.824725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:00.824725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:00.825014 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:00.825580 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:00.825599 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:00.825991 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 11:15:00.826118 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 11:15:00.826644 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":218,"thread_start_us":122,"threads_started":1}
I20260811 11:15:00.828028 24007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:00.828498 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.828339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46538)
0811 11:15:00.828398 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.828456 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 11:15:00.828739 23947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
I20260811 11:15:00.829291 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.829187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46540)
0811 11:15:00.829240 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.829273 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 11:15:00.829521 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:43175: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
Entry for principal kudu/127.29.145.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab.
I20260811 11:15:00.846065 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:43175
--webserver_interface=127.29.145.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:15:00.878865 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } attempt: 1
I20260811 11:15:00.879462 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.879308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46550)
0811 11:15:00.879388 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.879430 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 11:15:00.879829 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.879729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46556)
0811 11:15:00.879779 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.879804 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 11:15:00.879944 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:43175: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
W20260811 11:15:00.939262 24015 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:15:00.939558 24015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:15:00.939600 24015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:15:00.939661 24015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:15:00.941902 24015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:15:00.941988 24015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:15:00.942018 24015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:15:00.942040 24015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:15:00.942067 24015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:15:00.943866 24015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.61.keytab
--principal=kudu/127.29.145.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.61:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.29.145.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:15:00.944164 24015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:15:00.944412 24015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:15:00.948100 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } attempt: 2
I20260811 11:15:00.948616 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.948472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46566)
0811 11:15:00.948546 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.948585 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 11:15:00.949151 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.949048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46578)
0811 11:15:00.949102 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.949128 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 11:15:00.949254 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.61:43175: Network error: Client connection negotiation failed: client connection to 127.29.145.61:43175: connect: Connection refused (error 111)
W20260811 11:15:00.949554 24022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:15:00.949543 24024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:15:00.949847 24021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:15:00.950139 24015 server_base.cc:1034] running on GCE node
I20260811 11:15:00.950399 24015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:15:00.950716 24015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:15:00.951920 24015 hybrid_clock.cc:648] HybridClock initialized: now 1786446900951864 us; error 73 us; skew 500 ppm
Aug 11 11:15:00 dist-test-slave-69rg krb5kdc[23915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:15:00.954909 24015 init.cc:377] Logged in from keytab as kudu/127.29.145.61@KRBTEST.COM (short username kudu)
I20260811 11:15:00.955439 24015 webserver.cc:460] Webserver started at http://127.29.145.61:42407/ using document root <none> and password file <none>
I20260811 11:15:00.955694 24015 fs_manager.cc:359] Metadata directory not provided
I20260811 11:15:00.955766 24015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:15:00.955917 24015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:15:00.957015 24015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data/instance:
uuid: "cf05c1373eb64c00926b2a8221b78c7a"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "40f36beab5b785cc5651006dddc5afa7"
server_key_iv: "5da5785b413d8e97751c1f1a646e6316"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.957392 24015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal/instance:
uuid: "cf05c1373eb64c00926b2a8221b78c7a"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "40f36beab5b785cc5651006dddc5afa7"
server_key_iv: "5da5785b413d8e97751c1f1a646e6316"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.958967 24015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 11:15:00.960608 24033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:15:00.960901 24015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260811 11:15:00.961030 24015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal
uuid: "cf05c1373eb64c00926b2a8221b78c7a"
format_stamp: "Formatted at 2026-08-11 11:15:00 on dist-test-slave-69rg"
server_key: "40f36beab5b785cc5651006dddc5afa7"
server_key_iv: "5da5785b413d8e97751c1f1a646e6316"
server_key_version: "encryptionkey@0"
I20260811 11:15:00.961120 24015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:15:00.970611 24015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:15:00.973489 24015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:15:00.973723 24015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:15:00.980701 24015 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.61:43175
I20260811 11:15:00.980783 24092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.61:43175 every 8 connection(s)
I20260811 11:15:00.981287 24015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/data/info.pb
I20260811 11:15:00.982566 24093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:15:00.982980 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 24015
I20260811 11:15:00.983093 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-1/wal/instance
I20260811 11:15:00.983330 30276 external_mini_cluster.cc:1428] Setting key 6ad941c09f9dafe67c7b2a47f7ef858d
I20260811 11:15:00.985102 24093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:15:00 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.29.145.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.29.145.60@KRBTEST.COM" created.
I20260811 11:15:00.997474 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.986000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:40915 (local address 127.0.0.1:49036)
0811 11:15:00.986164 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.986208 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:00.986672 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:00.986932 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:00.986935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:00.987195 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:00.987390 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:00.987394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.988511 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.988513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:00.989326 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:00.989328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.990087 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.990089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:00.990319 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:00.991132 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:00.991142 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:00.992753 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:00.994811 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:00.994818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:00.994820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:00.995048 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:00.995988 (+ 940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:00.995989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:00.995990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:00.996109 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:00.996800 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:00.996809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:00.996954 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 11:15:00.997144 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 11:15:00.997295 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260811 11:15:00.997678 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.986020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49036 (local address 127.29.145.62:40915)
0811 11:15:00.986092 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:00.986095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:00.986303 (+ 208us) server_negotiation.cc:407] Connection header received
0811 11:15:00.986768 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:00.986770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:00.986796 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:00.986878 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:00.987491 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.988368 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.989428 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:00.989837 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:00.989925 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:00.992852 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:00.992854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:00.992855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:00.992862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:00.994696 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:00.995642 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:00.995643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:00.995643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:00.995705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:00.996517 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:00.996519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:00.996519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:00.996717 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:00.996809 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:00.997231 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 11:15:00.997493 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:15:00.998417 24093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:15:01.010717 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.998817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46586)
0811 11:15:00.999075 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:00.999088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:00.999165 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:00.999335 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:00.999336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:00.999483 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:00.999538 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:00.999540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.000850 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.000851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.001527 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.001531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.002303 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.002305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.002387 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.002834 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.002840 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.005039 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.008381 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.008384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.008385 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.008750 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.009560 (+ 810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.009561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.009562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.009637 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.010429 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.010430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.010473 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.010549 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.010640 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 11:15:01.010883 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:00.998917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.29.145.61:43175)
0811 11:15:00.999089 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:00.999095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:00.999174 (+ 79us) server_negotiation.cc:407] Connection header received
0811 11:15:00.999220 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:00.999223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:00.999237 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:00.999336 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:00.999624 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.000684 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.001643 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.002188 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.002298 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.005162 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.005173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.005173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.005181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.008140 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.008899 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.008903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.008903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.008979 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.009808 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.009810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.009810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.010313 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.010397 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.010725 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.010804 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260811 11:15:01.012321 24093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:01.012791 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.012644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35510)
0811 11:15:01.012718 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.012769 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 11:15:01.012976 24038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
I20260811 11:15:01.013950 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.013504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35520)
0811 11:15:01.013887 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.013923 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362}
W20260811 11:15:01.014497 24093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
Entry for principal kudu/127.29.145.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab.
I20260811 11:15:01.031847 30276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_kobm8/build/release/bin/kudu
/tmp/dist-test-task_kobm8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:38659
--webserver_interface=127.29.145.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.29.145.20:33889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:15:01.040431 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } attempt: 3
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:15:01.050803 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.041201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46590 (local address 127.29.145.61:43175)
0811 11:15:01.041293 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.041296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.041343 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:15:01.041510 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.041511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.041524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.041586 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.042294 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.042833 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.044440 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.044665 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.044735 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.047654 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.047656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.047657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.047674 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.049471 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.049937 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.049938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.049939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.050008 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.050221 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.050222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.050222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.050361 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.050438 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.050608 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.050694 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:15:01.050803 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.041125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46590)
0811 11:15:01.041224 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.041249 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.041379 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.041646 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.041648 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.041965 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.042068 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.042072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.042963 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.042966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.044284 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.044289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.044763 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.044764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.044851 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.045357 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.045363 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.047536 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.049586 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.049587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.049588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.049778 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.050083 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.050084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.050084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.050148 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.050442 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.050443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.050500 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.050568 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.050692 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 11:15:01.051262 24007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:01.051705 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.051590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35528)
0811 11:15:01.051653 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.051686 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 11:15:01.052178 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.051976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35530)
0811 11:15:01.052022 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.052053 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 11:15:01.052304 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
I20260811 11:15:01.063817 24093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 1
I20260811 11:15:01.064385 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.064216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35540)
0811 11:15:01.064290 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.064344 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 11:15:01.064890 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.064789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35550)
0811 11:15:01.064840 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.064870 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 11:15:01.065013 24093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
I20260811 11:15:01.083524 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 1
I20260811 11:15:01.084075 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.083937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35560)
0811 11:15:01.084003 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.084046 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:15:01.084465 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.084369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35564)
0811 11:15:01.084416 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.084443 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:15:01.084561 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
W20260811 11:15:01.130846 24101 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:15:01.131101 24101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:15:01.131150 24101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:15:01.131192 24101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:15:01.132346 24101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:15:01.132419 24101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:15:01.132452 24101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:15:01.132483 24101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:15:01.132510 24101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:15:01.133169 24093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 2
I20260811 11:15:01.133715 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.133554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35572)
0811 11:15:01.133633 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.133680 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 11:15:01.134155 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.134036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35576)
0811 11:15:01.134083 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.134112 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 11:15:01.134254 24093 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
I20260811 11:15:01.134538 24101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.29.145.20:33889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.29.145.62:40915,127.29.145.61:43175,127.29.145.60:38659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/krb5kdc/kudu_127.29.145.60.keytab
--principal=kudu/127.29.145.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.29.145.60:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.29.145.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 10:16:46 UTC on ccycloud.quasar-tsnumj.root.comops.site
build id 4580137
I20260811 11:15:01.134763 24101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:15:01.134963 24101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:15:01.140264 24107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:15:01.140441 24106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:15:01.140441 24110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:15:01.140838 24101 server_base.cc:1034] running on GCE node
I20260811 11:15:01.141088 24101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:15:01.141433 24101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:15:01.142642 24101 hybrid_clock.cc:648] HybridClock initialized: now 1786446901142625 us; error 34 us; skew 500 ppm
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446901, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:15:01.146059 24101 init.cc:377] Logged in from keytab as kudu/127.29.145.60@KRBTEST.COM (short username kudu)
I20260811 11:15:01.147009 24101 webserver.cc:460] Webserver started at http://127.29.145.60:43061/ using document root <none> and password file <none>
I20260811 11:15:01.147293 24101 fs_manager.cc:359] Metadata directory not provided
I20260811 11:15:01.147341 24101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:15:01.147437 24101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:15:01.148558 24101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data/instance:
uuid: "48202b45d5904e40a7fd72e763b16917"
format_stamp: "Formatted at 2026-08-11 11:15:01 on dist-test-slave-69rg"
server_key: "2e00627445652147c7164f6485f87c46"
server_key_iv: "d138b4378007210528f62743d5f7db85"
server_key_version: "encryptionkey@0"
I20260811 11:15:01.149039 24101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal/instance:
uuid: "48202b45d5904e40a7fd72e763b16917"
format_stamp: "Formatted at 2026-08-11 11:15:01 on dist-test-slave-69rg"
server_key: "2e00627445652147c7164f6485f87c46"
server_key_iv: "d138b4378007210528f62743d5f7db85"
server_key_version: "encryptionkey@0"
I20260811 11:15:01.150727 24101 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:15:01.151684 24117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:15:01.152025 24101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:15:01.152245 24101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data,/tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal
uuid: "48202b45d5904e40a7fd72e763b16917"
format_stamp: "Formatted at 2026-08-11 11:15:01 on dist-test-slave-69rg"
server_key: "2e00627445652147c7164f6485f87c46"
server_key_iv: "d138b4378007210528f62743d5f7db85"
server_key_version: "encryptionkey@0"
I20260811 11:15:01.152344 24101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:15:01.159730 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 2
I20260811 11:15:01.160619 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.160460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35592)
0811 11:15:01.160534 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.160577 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 11:15:01.161075 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.160987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35604)
0811 11:15:01.161032 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.161058 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 11:15:01.161178 24007 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.29.145.60:38659: Network error: Client connection negotiation failed: client connection to 127.29.145.60:38659: connect: Connection refused (error 111)
I20260811 11:15:01.174901 24101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:15:01.177510 24101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:15:01.177672 24101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:15:01.183197 24101 rpc_server.cc:307] RPC server started. Bound to: 127.29.145.60:38659
I20260811 11:15:01.183286 24175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.29.145.60:38659 every 8 connection(s)
I20260811 11:15:01.183663 24101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/data/info.pb
I20260811 11:15:01.184814 24176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:15:01.186522 24176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:01.189172 30276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_kobm8/build/release/bin/kudu as pid 24101
I20260811 11:15:01.189262 30276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_kobm8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786446611237378-30276-0/minicluster-data/master-2/wal/instance
I20260811 11:15:01.189463 30276 external_mini_cluster.cc:1428] Setting key 042a485e6f4f0b6ded3c654eafd2566c
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446901, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:15:01.197546 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.187485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.29.145.62:40915)
0811 11:15:01.187743 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.187753 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.187768 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:15:01.188238 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.188239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.188252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.188346 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.188994 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.189609 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.191495 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.191717 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.191801 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.194247 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.194248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.194249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.194256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.196072 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.196538 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.196540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.196540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.196604 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.196870 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.196871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.196872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.197001 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.197077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.197365 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.197446 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 11:15:01.197755 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.187278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.62:40915 (local address 127.0.0.1:49038)
0811 11:15:01.187603 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.187649 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.188123 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.188429 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.188433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.188693 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.188891 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.188894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.189720 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.189721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.191405 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.191408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.191827 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.191828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.191973 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.192519 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.192530 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.194144 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.196169 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.196175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.196178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.196465 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.196679 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.196680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.196680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.196795 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.197068 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.197075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.197217 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.197331 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.197509 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":244,"threads_started":1}
I20260811 11:15:01.198410 24176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:01.199239 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.189913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.29.145.60:38659)
0811 11:15:01.190099 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.190111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.190282 (+ 171us) server_negotiation.cc:407] Connection header received
0811 11:15:01.190362 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.190364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.190387 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.190554 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.191329 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.192037 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.193054 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.193386 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.193493 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.196095 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.196103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.196103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.196111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.197756 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.198217 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.198220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.198220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.198292 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.198486 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.198487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.198487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.198678 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.198805 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.198980 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.199121 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446901, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.62@KRBTEST.COM
I20260811 11:15:01.205421 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.198679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.61:43175 (local address 127.0.0.1:46600)
0811 11:15:01.198798 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.198809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.198857 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.199015 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.199016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.199211 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.199250 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.199252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.200214 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.200215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.200789 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.200791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.201121 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.201122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.201179 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.201610 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.201614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.202920 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.204466 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.204467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.204467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.204640 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.204851 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.204852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.204853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.204907 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.205187 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.205189 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.205234 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.205292 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.205364 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 11:15:01.205540 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.198752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.29.145.61:43175)
0811 11:15:01.198797 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.198799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.198843 (+ 44us) server_negotiation.cc:407] Connection header received
0811 11:15:01.198910 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.198911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.198921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.198970 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.199517 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.200140 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.200870 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.201052 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.201103 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.203009 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.203011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.203011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.203019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.204366 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.204717 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.204718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.204718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.204777 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.204977 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.204978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.204978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.205097 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.205180 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.205341 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.205419 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 11:15:01.206059 24176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:15:01.207734 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.200013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.29.145.62:40915)
0811 11:15:01.200049 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.200051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.200066 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:15:01.200082 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.200083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.200090 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.200142 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.200594 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.201160 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.201999 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.202250 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.202304 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.204769 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.204770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.204771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.204777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.206505 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.206985 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.206986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.206987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.207050 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.207238 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.207239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.207239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.207368 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.207508 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.207543 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.207616 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446901, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.60@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.29.145.61@KRBTEST.COM
I20260811 11:15:01.213559 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.206334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35622)
0811 11:15:01.206383 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.206393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.206433 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.206560 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.206561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.206711 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.206749 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.206751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.207831 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.207832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.208566 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.208569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.208922 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.208923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.208978 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.209391 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.209396 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.210691 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.212374 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.212374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.212375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.212588 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.212817 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.212818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.212818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.212870 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.213205 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.213206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.213257 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.213352 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.213461 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 11:15:01.213799 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.206369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.29.145.60:38659)
0811 11:15:01.206408 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.206410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.206422 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:15:01.206474 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.206475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.206486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.206548 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.206985 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.207718 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.208643 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.208846 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.208897 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.210771 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.210772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.210772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.210779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.212280 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.212685 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.212687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.212687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.212749 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.212940 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.212941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.212942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.213080 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.213170 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.213401 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.213559 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":24}
I20260811 11:15:01.215767 24176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: Bootstrap starting.
I20260811 11:15:01.216655 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.208307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.29.145.61:43175)
0811 11:15:01.208341 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.208343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.208353 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:15:01.208373 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.208374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.208388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.208426 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.208958 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.209376 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.210205 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.210522 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.210585 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.213209 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.213210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.213210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.213219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.215341 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.215704 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.215705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.215706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.215777 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.216089 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.216090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.216090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.216259 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.216451 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.216492 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.216575 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 11:15:01.216924 24176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: Neither blocks nor log segments found. Creating new log.
I20260811 11:15:01.217213 24176 log.cc:824] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: Log is configured to *not* fsync() on all Append() calls
I20260811 11:15:01.218289 24176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: No bootstrap required, opened a new log
I20260811 11:15:01.218551 30276 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:15:01.219854 24176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.220014 24176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48202b45d5904e40a7fd72e763b16917, State: Initialized, Role: FOLLOWER
I20260811 11:15:01.220109 24176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.219568 23975 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f387c9464e274175bb8458eb5059ec27"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49052
I20260811 11:15:01.220500 24176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:15:01.220476 24183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.220570 24183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:15:01.222405 24197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:15:01.222476 24197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:15:01.222767 24101 master_runner.cc:186] Error getting master registration for 127.29.145.62:40915: OK, instance_id { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" instance_seqno: 1786446900808071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:15:01.224843 24101 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:15:01.225472 24093 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 3
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.61@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:15:01.233393 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.225862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35624)
0811 11:15:01.225923 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.225937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.225993 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.226170 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.226171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.226336 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.226382 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.226383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.227052 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.227053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.227614 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.227616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.228005 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.228006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.228063 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.228616 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.228622 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.230181 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.232180 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.232181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.232182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.232412 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.232641 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.232642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.232643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.232711 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.233054 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.233055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.233103 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.233173 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.233280 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 11:15:01.233399 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.225973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.29.145.60:38659)
0811 11:15:01.226016 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.226019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.226027 (+ 8us) server_negotiation.cc:407] Connection header received
0811 11:15:01.226050 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.226051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.226062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.226113 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.226465 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.226960 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.227707 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.227927 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.228007 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.230283 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.230284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.230285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.230293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.232081 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.232497 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.232499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.232499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.232565 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.232781 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.232782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.232782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.232977 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.233056 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.233219 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.233304 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 11:15:01.235455 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: Bootstrap starting.
I20260811 11:15:01.236145 24093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: Neither blocks nor log segments found. Creating new log.
I20260811 11:15:01.236446 24093 log.cc:824] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:15:01.237229 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: No bootstrap required, opened a new log
I20260811 11:15:01.238816 24093 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.238966 24093 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf05c1373eb64c00926b2a8221b78c7a, State: Initialized, Role: FOLLOWER
I20260811 11:15:01.239089 24093 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.239485 24093 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:15:01.239445 24201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.239540 24201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:15:01.241418 24214 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:15:01.241482 24214 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:15:01.241727 24015 master_runner.cc:186] Error getting master registration for 127.29.145.62:40915: OK, instance_id { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" instance_seqno: 1786446900808071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:15:01.244642 24015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:15:01.260315 24007 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } attempt: 3
Aug 11 11:15:01 dist-test-slave-69rg krb5kdc[23915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786446900, etypes {rep=17 tkt=17 ses=17}, kudu/127.29.145.62@KRBTEST.COM for kudu/127.29.145.60@KRBTEST.COM
I20260811 11:15:01.268175 24012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.260714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.29.145.60:38659 (local address 127.0.0.1:35640)
0811 11:15:01.260783 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 11:15:01.260796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:15:01.260846 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:15:01.261009 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:15:01.261011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:15:01.261229 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:15:01.261286 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.261288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.262227 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.262228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.262744 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:15:01.262746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.263295 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.263296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:15:01.263360 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.263862 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:15:01.263867 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:15:01.265382 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:15:01.267050 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.267051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.267052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.267276 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.267476 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:15:01.267477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:15:01.267478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:15:01.267539 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:15:01.267846 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:15:01.267847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:15:01.267893 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 11:15:01.267956 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:15:01.268065 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 11:15:01.268175 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.260782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.29.145.60:38659)
0811 11:15:01.260850 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.260853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.260884 (+ 31us) server_negotiation.cc:407] Connection header received
0811 11:15:01.260914 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.260915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.260933 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.261003 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.261373 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.262148 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.262845 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.263163 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.263232 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.265483 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.265484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.265485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.265492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.266940 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.267350 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.267351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.267351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.267408 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.267599 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.267601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.267601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.267755 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.267818 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.268008 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.268080 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:15:01.270098 24007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27: Bootstrap starting.
I20260811 11:15:01.270869 24007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27: Neither blocks nor log segments found. Creating new log.
I20260811 11:15:01.271184 24007 log.cc:824] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27: Log is configured to *not* fsync() on all Append() calls
I20260811 11:15:01.272059 24007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27: No bootstrap required, opened a new log
I20260811 11:15:01.273466 24007 raft_consensus.cc:348] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.273617 24007 raft_consensus.cc:729] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f387c9464e274175bb8458eb5059ec27, State: Initialized, Role: FOLLOWER
I20260811 11:15:01.273705 24007 consensus_queue.cc:260] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.274078 24007 sys_catalog.cc:564] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:15:01.274103 23975 raft_consensus.cc:482] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 11:15:01.274158 23975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:15:01.275079 23975 raft_consensus.cc:504] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.275372 23975 leader_election.cc:288] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [CANDIDATE]: Term 1 election: Requested vote from peers cf05c1373eb64c00926b2a8221b78c7a (127.29.145.61:43175), 48202b45d5904e40a7fd72e763b16917 (127.29.145.60:38659)
I20260811 11:15:01.276094 24065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f387c9464e274175bb8458eb5059ec27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cf05c1373eb64c00926b2a8221b78c7a"
I20260811 11:15:01.276263 24065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:15:01.277576 24065 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f387c9464e274175bb8458eb5059ec27 in term 1.
I20260811 11:15:01.277863 24145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f387c9464e274175bb8458eb5059ec27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "48202b45d5904e40a7fd72e763b16917"
I20260811 11:15:01.277998 24145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:15:01.278258 23947 leader_election.cc:302] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf05c1373eb64c00926b2a8221b78c7a, f387c9464e274175bb8458eb5059ec27; no voters:
I20260811 11:15:01.278982 24145 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f387c9464e274175bb8458eb5059ec27 in term 1.
I20260811 11:15:01.274041 24217 sys_catalog.cc:455] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.279637 24217 sys_catalog.cc:458] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:15:01.279753 24217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:15:01.279875 24217 raft_consensus.cc:686] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [term 1 LEADER]: Becoming Leader. State: Replica: f387c9464e274175bb8458eb5059ec27, State: Running, Role: LEADER
I20260811 11:15:01.280025 24217 consensus_queue.cc:237] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } }
I20260811 11:15:01.280304 23929 master_runner.cc:263] No leader master found from master f387c9464e274175bb8458eb5059ec27
I20260811 11:15:01.280355 23929 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:15:01.280700 24217 sys_catalog.cc:455] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f387c9464e274175bb8458eb5059ec27. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.280874 24217 sys_catalog.cc:458] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: This master's current role is: LEADER
I20260811 11:15:01.281793 24236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:15:01.282409 24236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:15:01.284015 24145 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [term 1 FOLLOWER]: Refusing update from remote peer f387c9464e274175bb8458eb5059ec27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:15:01.284078 24065 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [term 1 FOLLOWER]: Refusing update from remote peer f387c9464e274175bb8458eb5059ec27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:15:01.284323 24234 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:15:01.284451 24217 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:15:01.286140 24183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f387c9464e274175bb8458eb5059ec27. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.286252 24183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:15:01.286912 24201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f387c9464e274175bb8458eb5059ec27. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.287007 24201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:15:01.294234 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.281569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.29.145.60:38659)
0811 11:15:01.281619 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.281622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.283330 (+ 1708us) server_negotiation.cc:407] Connection header received
0811 11:15:01.283436 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.283438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.283449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.283479 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.286427 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.287154 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.287878 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.288090 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.288182 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.289467 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.289469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.289469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.289476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.292037 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.293035 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.293037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.293037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.293098 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.293292 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.293292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.293293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.293418 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.293493 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.294034 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.294122 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:15:01.299171 24010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.282079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.29.145.62:40915)
0811 11:15:01.282127 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.282129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.285230 (+ 3101us) server_negotiation.cc:407] Connection header received
0811 11:15:01.285433 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.285434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.285445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.285475 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.290542 (+ 5067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.291305 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.293068 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.293376 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.293418 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.295899 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.295901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.295902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.295910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.297729 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.298225 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.298226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.298227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.298288 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.298536 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.298537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.298538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.298691 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.298786 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.299003 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.299079 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 11:15:01.299335 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.299499 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: This master's current role is: LEADER
I20260811 11:15:01.299548 24201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.299654 24201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:15:01.299775 24243 mvcc.cc:205] Tried to move back new op lower bound from 7317286507657105408 to 7317286507644510208. Current Snapshot: MvccSnapshot[applied={T|T < 7317286507657105408}]
I20260811 11:15:01.299858 24246 mvcc.cc:205] Tried to move back new op lower bound from 7317286507657105408 to 7317286507644510208. Current Snapshot: MvccSnapshot[applied={T|T < 7317286507657105408}]
I20260811 11:15:01.300578 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:01.282453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.29.145.61:43175)
0811 11:15:01.282521 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.282523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.289976 (+ 7453us) server_negotiation.cc:407] Connection header received
0811 11:15:01.290069 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.290070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.290083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.290144 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.291039 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.291735 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.294530 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:01.294814 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:01.294858 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:01.296662 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:15:01.296664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:15:01.296664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:15:01.296672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:15:01.298772 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.299671 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.299672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.299673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.299751 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:15:01.299980 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:15:01.299981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:15:01.299981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:15:01.300115 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:15:01.300189 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:01.300409 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 11:15:01.300490 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:15:01.301503 24217 sys_catalog.cc:455] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.301621 24217 sys_catalog.cc:458] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27 [sys.catalog]: This master's current role is: LEADER
I20260811 11:15:01.302042 24183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f387c9464e274175bb8458eb5059ec27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f387c9464e274175bb8458eb5059ec27" member_type: VOTER last_known_addr { host: "127.29.145.62" port: 40915 } } peers { permanent_uuid: "cf05c1373eb64c00926b2a8221b78c7a" member_type: VOTER last_known_addr { host: "127.29.145.61" port: 43175 } } peers { permanent_uuid: "48202b45d5904e40a7fd72e763b16917" member_type: VOTER last_known_addr { host: "127.29.145.60" port: 38659 } } }
I20260811 11:15:01.302126 24183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:15:01.310395 24236 catalog_manager.cc:1347] Generated new cluster ID: f79f99e0d70b4e81bce6f35374d54574
I20260811 11:15:01.310467 24236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:15:01.359612 24236 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:15:01.360445 24236 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:15:01.374646 24236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f387c9464e274175bb8458eb5059ec27: Generated new TSK 0
I20260811 11:15:01.375114 24236 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:15:01.409252 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.407690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.29.145.60:38659)
0811 11:15:01.407753 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.407756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.408467 (+ 711us) server_negotiation.cc:407] Connection header received
0811 11:15:01.408596 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.408598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.408611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.408648 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.409174 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35656: BlockingRecv error: recv got EOF from 127.0.0.1:35656 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 11:15:01.410403 24010 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.408133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.29.145.62:40915)
0811 11:15:01.408180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.408182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.409751 (+ 1569us) server_negotiation.cc:407] Connection header received
0811 11:15:01.409916 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.409917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.409930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.409964 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.410329 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49080: BlockingRecv error: recv got EOF from 127.0.0.1:49080 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 11:15:01.411700 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.410933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.29.145.61:43175)
0811 11:15:01.410982 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.410984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.411125 (+ 141us) server_negotiation.cc:407] Connection header received
0811 11:15:01.411230 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.411231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.411243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.411275 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.411628 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46630: BlockingRecv error: recv got EOF from 127.0.0.1:46630 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 11:15:01.414970 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.414108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.29.145.61:43175)
0811 11:15:01.414148 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.414150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.414365 (+ 215us) server_negotiation.cc:407] Connection header received
0811 11:15:01.414485 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.414486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.414505 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.414538 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.414911 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46634: BlockingRecv error: recv got EOF from 127.0.0.1:46634 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 11:15:01.418599 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.417576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.29.145.61:43175)
0811 11:15:01.417620 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.417622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.417947 (+ 325us) server_negotiation.cc:407] Connection header received
0811 11:15:01.418130 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.418132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.418144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.418177 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.418546 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46646: BlockingRecv error: recv got EOF from 127.0.0.1:46646 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 11:15:01.423202 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.422230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.29.145.61:43175)
0811 11:15:01.422271 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.422273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.422562 (+ 289us) server_negotiation.cc:407] Connection header received
0811 11:15:01.422675 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.422676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.422688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.422721 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.423147 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46656: BlockingRecv error: recv got EOF from 127.0.0.1:46656 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 11:15:01.429672 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.428738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.29.145.61:43175)
0811 11:15:01.428783 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.428785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.429060 (+ 275us) server_negotiation.cc:407] Connection header received
0811 11:15:01.429172 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.429174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.429187 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.429220 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.429613 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46660: BlockingRecv error: recv got EOF from 127.0.0.1:46660 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 11:15:01.440104 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.439322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.29.145.61:43175)
0811 11:15:01.439371 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.439373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.439474 (+ 101us) server_negotiation.cc:407] Connection header received
0811 11:15:01.439566 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.439567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.439582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.439662 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.440037 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46666: BlockingRecv error: recv got EOF from 127.0.0.1:46666 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 11:15:01.458175 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.457381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.29.145.61:43175)
0811 11:15:01.457465 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.457467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.457505 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:15:01.457588 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.457589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.457603 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.457697 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.458109 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46670: BlockingRecv error: recv got EOF from 127.0.0.1:46670 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:15:01.492238 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.491434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.29.145.61:43175)
0811 11:15:01.491515 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.491517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.491571 (+ 54us) server_negotiation.cc:407] Connection header received
0811 11:15:01.491624 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.491625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.491637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.491690 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.492169 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46678: BlockingRecv error: recv got EOF from 127.0.0.1:46678 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:15:01.558586 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.557692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.29.145.61:43175)
0811 11:15:01.557806 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.557809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.557931 (+ 122us) server_negotiation.cc:407] Connection header received
0811 11:15:01.558031 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.558032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.558044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.558135 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.558507 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46680: BlockingRecv error: recv got EOF from 127.0.0.1:46680 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 11:15:01.688870 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.688008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.29.145.61:43175)
0811 11:15:01.688071 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.688073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.688164 (+ 91us) server_negotiation.cc:407] Connection header received
0811 11:15:01.688259 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.688260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.688271 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.688324 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.688798 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46684: BlockingRecv error: recv got EOF from 127.0.0.1:46684 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 11:15:01.946977 24098 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:15:01.946192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.29.145.61:43175)
0811 11:15:01.946269 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:01.946271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:01.946324 (+ 53us) server_negotiation.cc:407] Connection header received
0811 11:15:01.946352 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:01.946353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:01.946365 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:01.946408 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:15:01.946891 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46700: BlockingRecv error: recv got EOF from 127.0.0.1:46700 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:15:02.223119 24197 catalog_manager.cc:1259] Loaded cluster ID: f79f99e0d70b4e81bce6f35374d54574
I20260811 11:15:02.223258 24197 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: loading cluster ID for follower catalog manager: success
I20260811 11:15:02.224253 24197 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: acquiring CA information for follower catalog manager: success
I20260811 11:15:02.224645 24197 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 48202b45d5904e40a7fd72e763b16917: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:15:02.242096 24214 catalog_manager.cc:1259] Loaded cluster ID: f79f99e0d70b4e81bce6f35374d54574
I20260811 11:15:02.242197 24214 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: loading cluster ID for follower catalog manager: success
I20260811 11:15:02.243186 24214 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: acquiring CA information for follower catalog manager: success
I20260811 11:15:02.243597 24214 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf05c1373eb64c00926b2a8221b78c7a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:15:02.464443 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:02.460657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.29.145.61:43175)
0811 11:15:02.460861 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:02.460864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:02.460888 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:15:02.460959 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:02.460961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:02.461045 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:02.461164 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:15:02.461464 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.462124 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.463211 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.463449 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.463514 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:02.463893 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:15:02.464041 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:15:02.464121 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:02.464245 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 11:15:02.464276 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
I20260811 11:15:02.469764 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:02.467111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.29.145.60:38659)
0811 11:15:02.467233 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:02.467235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:02.467242 (+ 7us) server_negotiation.cc:407] Connection header received
0811 11:15:02.467288 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:02.467289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:02.467335 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:02.467456 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:15:02.467570 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.468084 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.468851 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.469071 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.469151 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:02.469324 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:15:02.469459 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:15:02.469549 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:02.469593 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 11:15:02.469624 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260811 11:15:02.475497 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:02.471920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.29.145.60:38659)
0811 11:15:02.471963 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:02.471965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:02.472199 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:15:02.472321 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:02.472322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:02.472335 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:02.472449 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:15:02.472671 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.473497 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474645 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.474948 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474988 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:02.475175 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:15:02.475213 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:15:02.475268 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:02.475360 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 11:15:02.475389 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 11:15:02.475543 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:02.472065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.29.145.62:40915)
0811 11:15:02.472197 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:02.472199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:02.472209 (+ 10us) server_negotiation.cc:407] Connection header received
0811 11:15:02.472303 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:02.472305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:02.472372 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:02.472463 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:15:02.472656 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.473186 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474318 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.474737 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474831 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:02.475088 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:15:02.475236 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:15:02.475318 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:02.475379 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:15:02.475412 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260811 11:15:02.475703 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:15:02.471993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.29.145.61:43175)
0811 11:15:02.472044 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 11:15:02.472046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 11:15:02.472186 (+ 140us) server_negotiation.cc:407] Connection header received
0811 11:15:02.472360 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:15:02.472361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:15:02.472400 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:15:02.472470 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:15:02.472628 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.473519 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474574 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:15:02.474881 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:15:02.474961 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:15:02.475308 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:15:02.475362 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:15:02.475452 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:15:02.475548 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 11:15:02.475580 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 11:15:02.478722 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 23929
I20260811 11:15:02.494041 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 24015
I20260811 11:15:02.504307 30276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_kobm8/build/release/bin/kudu with pid 24101
2026-08-11T11:15:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3096 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6056 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291280 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST